2026-09-10 21:24 UTC
@antoinechambertloir@mathstodon.xyz
I'm sorry to put some salt in this thread; I want to correct one point. Describing LLMs as operating according to "probabilistic rules" and producing "randomly generated text" is misleading.
The internal workings of an LLM do not look like a set of logical or probabilistic rules. Given an input, the model _deterministically_ computes scores (called logits) for the possible next tokens (e.g. words, punctuation). These scores are then converted into a probability distribution.
A separate decoding step selects the next token, either deterministically or by sampling: that is, by drawing a token according to that probability distribution.
Replies (1)
-
@antoinechambertloir@mathstodon.xyz 2026-09-10 21:45
@0xdefec7edcafe@piaille.fr this sounds compatible with what I wrote, except that I didn't make clear that the probabilistic rules evolve during the production of the text, and already after the prompt, it's not just esantirulo once and for all.