2026-04-08 15:18 UTC
@mirth @alienghic @cwebber
That's the thing, though, I don't think even the smallest trend is there. It's not that "it's early yet, but it's pointing in the right direction." I haven't seen anything that suggests there's any downward trend at all, no matter how small.
To get anything like equivalent performance out of the discount models like Deepseek, you have to run multiple instances in parallel or run a bunch of agents along with it.
The big breakthroughs in capability in the last year or two have all been about ramping *up* the power usage, model size, and GPU capacity, either by using bigger token windows or adding secondary agents. That's kind of some cool engineering, but power consumption or operating costs are just going up.
The main advantage of the discount models is that they have a much lower cost of training, mostly because (we suspect) they were trained off of responses from the bigger models.
Replies (1)
-
@mirth@mastodon.sdf.org 2026-04-08 20:14
@MichaelTBacon @alienghic @cwebber That's not consistent with what I've seen. Using weight footprint as a rough guide, the capability at the 1B, 10B, 30B, etc sizes continue to increase, and current smaller models perform better than larger predecessors. True on the closed side as well: Today's Sonnet is stronger than last year's Sonnet _and_ Opus, today's GPT-5.4 mini is stronger than GPT-5 at release, etc. Current 30B tier open models are stronger than GPT-4 was.