2026-04-08 20:45 UTC
@MichaelTBacon @alienghic @cwebber This is a synthesis of what I've seen across years of doing compute related work plus reading the published benchmark data and some papers. In my opinion it is necessary to have a reasonable knowledge of at least what's been published before having specific opinions about what the technical trends are or are not. Here's a leaderboard for SWE Bench Verified, a reasonable gauge of one dimension of model strength:
https://llm-stats.com/benchmarks/swe-bench-verified
Replies (2)
-
@mirth@mastodon.sdf.org 2026-04-08 20:48
@MichaelTBacon @alienghic @cwebber If you want an experiential reference for compute efficiency you can try https://chatjimmy.ai. It's using an old model, but the speed can give you some idea of what's possible in the next generation of silicon.
-
@MichaelTBacon@social.coop 2026-04-08 21:01
@mirth @alienghic @cwebber What is the origin of the cost value in that column?