It’s temperature primarily. That being said there is still a chance that an LLM can output values that are unexpected even at low temperatures.
Junkasaurus
joined 1 month ago
What are you saying precisely? It’s well known that LLMs have non-deterministic output (Ilya Sutskever even claims as such). Are you saying the way it goes about retrieving tokens as deterministic?
I can agree with that largely but I still contend you’re conflating a few things to make that argument. Fundamentally an LLM will make predictions based on probability (ignoring temperature) and probability does not equal certainty.