Elektrine lite

← Feed

@BioMan@awful.systems

2026-11-11 07:14 UTC

But they really aren’t! The actually interesting scaling laws from about 5-7 years ago that opened up machine learning to more applications all show that everything about machine learning is power-law diminishing returns with linear improvements in any metric requiring exponential increases in numbers of parameters / amounts of data / number of backprop cycles applied.

Replies (2)

  • @samvines@awful.systems 2026-11-11 09:03

    The scaling laws maxed out around mid 2023 with the release of GPT-4 - that was around the time that everyone got excited about “test time inference” which is where LLMs spit out reasoning streams to convince you they are “thinking”. If you look at where LLMs have improved since then it’s mainly in tasks where they can be strapped into a harness and given concrete feedback like “your program didn’t compile” which are also coincidentally use cases where data from gpt and Claude users can easily be harvested and new training data can be squeezed out of underpaid third world mturkers. The big labs have just been doing an incredible job of marketing (and particularly in Anthropic’s case, brain damaging themselves into believing their own marketing)

    Open ##5518368

  • @Amoeba_Girl@awful.systems 2026-11-11 15:43

    uh but have you considered unbounded recursive self-improvement? it’s how it works in books and in my imagination

    Open ##5519648