2026-08-23 17:59 UTC
😀
I know very well (and faced several times) curve overfitting: it's a term related to models working very well on the "training data", but badly on new samples outside that set.
Using the term for LLM output reproducing verbatim fragments of the training data is just mischaracterizing the term to fool those who do not understand statistical algorithms (and neural networks in particular).
As for Shannon, you seem to ignore what lossy compression is: https://en.wikipedia.org/wiki/Lossy_compression
Having said that, I'm afraid that no amount of computer science or examples can change the mind of an #AI believer/addict.
Replies (1)
-
@troed@swecyb.com 2026-08-23 18:12
@giacomo Let me guess - in your worldview anyone who does not agree with your definitions is wrong? Tell me - what would the compression ratio be for a 12B model to accurately contain all possible pictures of signs and notes in the world? https://www.ibm.com/docs/en/watsonx/saas?topic=atlas-overfitting