2026-04-08 20:17 UTC
@MichaelTBacon @alienghic @cwebber That's models. On the silicon side, ten years ago the bulk of inference ran on regular CPUs. Most of that moved to GPUs maybe five years ago, and now specialist chips are becoming more common. I know a little bit about chip design, and there's a lot of inefficiency that will get designed out of accelerators (reliance on DRAM is the big one, Cerebras and Taalas are the two notable current attempts).
Replies (1)
-
@mirth@mastodon.sdf.org 2026-04-08 20:23
@MichaelTBacon @alienghic @cwebber That's evidence of a downward trend in model size for any capability, and a downward trend in the cost to run a model of a given size. It seems likely that both continue for at least a few more years, and the consequence of that is Sonnet-level models available in pocket-size devices. How do we manage the fallout from that, and steer away from some of the worst downstream effects?