Skip to content
All apps

The temperature dial

About this app

Logits become a distribution via softmax(z/T): temperature, greedy, sampling, top-k and top-p with a visible cut-off line, entropy, a decision tree of the generation and loops at T = 0.

From logits to probabilities

A language model outputs a number for every possible next token, the logit z. The softmax turns these into probabilities. Temperature T divides all logits first: p = softmax(z/T). As T → 0 all the mass goes to the top token, T = 1 leaves the distribution unchanged, and a large T flattens it; the ranking always stays the same. A decoding strategy then chooses: greedy always takes the most likely token, sampling draws from the whole distribution, top-k only from the k best, and top-p (nucleus sampling) from the smallest top group whose mass is at least p.

What the app shows

For five prompts such as “Once upon a time there was a”, the app shows twelve candidates with logit and probability. A dashed line marks where top-k or top-p cut off, and a mass strip shows the random number drawn. Entropy H(p), the effective choice 2^H and the mass in the pool are shown as well. On the right the text grows token by token, and a tree shows the alternatives that were not chosen. The logits are hand-made for illustration; a real model has tens of thousands of candidates.

Things to try

Move the temperature between 0 and 3 and watch the bars get steeper or flatter without changing their order. Switch between greedy, sampling, top-k and top-p and set k or p. Click an alternative in the tree to branch off and see how an early choice changes the rest. The compare button puts greedy at T = 0, top-p 0.9 at T = 0.7 and sampling at T = 1 and T = 2.2 side by side, including a repetition loop and off-track tokens.

Frequently asked questions

What does temperature mean in ChatGPT and other LLMs?
Temperature controls how randomly a language model picks the next token. The logits are divided by T before the softmax: low values make the distribution peaked and the output predictable, high values make it flat and the output more varied but also more error-prone. The order of the candidates does not change.
What is the difference between top-k and top-p?
Top-k always samples from a fixed number k of the most likely tokens. Top-p, also called nucleus sampling, takes the smallest group of most likely tokens whose probabilities add up to at least p. This lets the size of the pool adapt to the shape of the distribution.
Why does greedy decoding often repeat itself?
Greedy decoding always picks the most likely token, so it is fully deterministic. Once the text reaches a sequence after which the same continuation is again the most likely, it gets stuck in a loop. Sampling with a moderate temperature or top-p tends to break such repetitions.
What can I try in this app?
You set temperature, strategy, k and p and draw one token at a time. A decision tree shows the alternatives and lets you branch off, and the comparison shows four settings side by side for the same prompt.

Subject: Neural networks

More from Neural networks

A VisuApp by heyprof: interactive, free in your browser, no sign-up. What is a VisuApp? · All apps