2026-10-17 16:54 UTC
Nearly all evaluation steps for an LLM I produce some level of entropy. Yes you can evaluate an LLM model deterministically, but we don’t. We even tune the levels to get good results. Model cards have sampling parameters supplied to give “the best” results. For example, Qwen 3.8 flash next gives
We recommend using the following sets of sampling parameters for generation:
Thinking Mode: temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=0.0, repetition_penalty=1.0
Instruct (or non-thinking) mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0
Replaying the seed will only get you the same result using a particular engine and a particular random number generator. Run it on llama.cpp Vs Strata and you can get different results. Different libc, different results.
A compiler is systematically deterministic. No calls to rand()
Replies (1)
-
@hirihit640@sh.itjust.works 2026-10-17 20:51
Replaying the seed will only get you the same result using a particular engine and a particular random number generator And the seed and rng are all part of the LLM. If you use the same inputs you get the same output. That’s the definition of determinism. Determinism does not care whether or not the user/programmer understands the program or not. The word you’re probably looking for is “unpredictable” or “less predictable”.