Temperature

You run the same prompt twice and get two slightly different answers. You run it again at a different time and get a third. This isn't a bug — it's the sampling step in generation doing exactly what it's designed to do. The parameter that controls how much variation you get is called temperature, and knowing when to crank it up and when to pin it at zero is the difference between reliable production code and a feature that works in the demo and mysteriously breaks on Thursday.

Generation, briefly

You met this in an earlier lesson, but the three steps are worth keeping in the front of your head:

1. Tokenisation — the model breaks the input into tokens.
2. Prediction — it computes a probability for every possible next token.
3. Sampling — it picks one based on those probabilities and repeats.

Temperature lives in step three. It changes how the model picks from the probability distribution, not what that distribution looks like.

What temperature actually does

Temperature is a decimal between 0 and 1 that reshapes the probability distribution before sampling. Low values make the distribution peakier — the highest-probability token dominates and is almost always chosen. High values flatten the distribution, pushing more probability mass onto lower-ranked tokens and making the sampler more willing to pick them.

Two things to internalise:

• Low temperature ≠ guaranteed identical output. The distribution is still probabilistic. Near-zero temperatures are close to deterministic in practice but not bit-for-bit reproducible.
• High temperature ≠ random gibberish. The model still respects the overall distribution — you get more variation, not nonsense.

The use-case mapping

| Temperature | Use for | Why | |---|---|---| | 0.0 – 0.3 (low) | Factual Q&A, coding, data extraction, classification, structured output | You want the best answer every time, not a creative one | | 0.4 – 0.7 (medium) | Summarisation, educational content, problem-solving, explanations | A little variation helps, but accuracy still matters | | 0.8 – 1.0 (high) | Brainstorming, creative writing, marketing copy, idea generation | Variety is the product; determinism defeats the point |

Default posture: if you're not sure, start at 0.0 for anything where there's a right answer and 0.7 for anything where there isn't. Adjust from there based on eval results — which, not coincidentally, is the topic of the very next module.

Adding temperature to your chat helper

Temperature is a top-level parameter on client.messages.create(), so it slots cleanly into the helper you've been building:

def chat(messages, system=None, temperature=1.0):
    params = {
        "model": model,
        "max_tokens": 1000,
        "messages": messages,
        "temperature": temperature,
    }
    if system:
        params["system"] = system
    message = client.messages.create(**params)
    return message.content[0].text

A few notes on this shape:

• temperature has a default, so existing call sites keep working.
• Unlike system, temperature can be set directly — there's no None-related footgun.
• Callers now write chat(messages, temperature=0.0) for a factual call or chat(messages, temperature=0.9) for a brainstorming session.

The production rule of thumb: pick a temperature for each endpoint or feature based on what that endpoint is doing, and keep it fixed. Don't surface temperature as a user-facing control unless you have a strong reason — users don't want a slider, they want good output. Your job is to know the right temperature for the job.