Temperature is Temperature is a hyperparameter that controls the randomness or creativity of a language model's output, scaling the probability distribution of predicted tokens before sampling.. In the context of ai,
it refers to In AI and large language models, temperature determines how deterministic or exploratory the model's responses are—lower values produce consistent, focused outputs while higher values increase diversity and unpredictability..
How Temperature Works
Temperature scales the logits (raw prediction scores) before the softmax function converts them to probabilities. At temperature 1.0, probabilities remain unchanged. Below 1.0, high-probability tokens become more dominant, reducing variation. Above 1.0, the distribution flattens, giving lower-probability tokens more selection chances.
Temperature Examples
- At temperature 0.3 with the prompt ‘The capital of France is', the model will almost always output ‘Paris' because the probability distribution heavily favors the most likely token.
- At temperature 1.5 with the same prompt, the model might occasionally output ‘Lyon' or ‘Marseille' because the flattened distribution allows lower-probability alternatives a realistic chance of selection.
- In creative writing tasks, temperature 0.7-0.9 balances coherence with variation—the model maintains logical structure while introducing enough novelty to avoid repetitive outputs.
Why Temperature Matters
Temperature directly impacts output quality and usefulness depending on the task. For factual queries and code generation, low temperature ensures reliability. For brainstorming and creative tasks, moderate-to-high temperature produces diverse, useful alternatives without becoming incoherent.
Common Mistakes with Temperature
- Setting temperature to 0 (which technically disables sampling and uses argmax selection) and expecting it to behave identically across all inference engines—different implementations handle zero-temperature differently.
- Using high temperature (above 2.0) for knowledge-intensive tasks, resulting in hallucinations and factually incorrect responses because the model equally weights unlikely tokens.
- Assuming temperature is universal across all models—different architectures and training methods respond differently to the same temperature value, requiring empirical testing.
Related Terms
Frequently Asked Questions
What does Temperature mean?
Temperature is a parameter ranging from 0 to 2+ that controls output randomness. At 0, the model picks the highest-probability token every time. At 1.0, probabilities are unmodified. Above 1.0, lower-probability tokens get boosted chances, increasing variety.
Why is Temperature important?
Temperature is critical because it determines whether your model produces reliable, consistent answers (low temperature) or diverse, creative responses (high temperature). The right setting depends entirely on your task requirements.
How do I use Temperature?
Adjust temperature through your API call or inference parameters. Start with 0.7 for balanced performance, decrease to 0.3-0.5 for factual accuracy, and increase to 0.9-1.2 for creative tasks. Test incrementally to find your optimal value.
