Chain of Thought: Why Working It Out Helps
Before reasoning models existed, people discovered a simple trick. If you added "Let's think step by step" to a math question, many models got the answer right far more often. Nothing about the model changed. It just wrote out its steps before answering.
That idea is called chain of thought, and it is the foundation of every reasoning model. This lesson explains why writing intermediate steps helps a language model so much, using a simple example.
What You'll Learn
- What chain of thought is, with a worked example
- Why intermediate steps help a model that predicts one token at a time
- How chain of thought went from a prompting trick to a built-in behavior
- Why reasoning models need less step-by-step prompting from you
A small example
Here is a question that trips up fast answers:
A shop sells notebooks for 3 each, or a pack of 4 for 9. You need 7 notebooks. What is the best way to buy them?
A model answering instantly might say "one pack and three singles, for 18." That is cheap, but it is not the whole story. It pattern-matched to "buy exactly what you need" and stopped.
With chain of thought, the model writes something like:
- Seven singles: 7 × 3 = 21.
- One pack plus three singles: 9 + 9 = 18, for exactly 7 notebooks.
- Two packs: 9 + 9 = 18, for 8 notebooks.
- Options 2 and 3 cost the same, but option 3 gives one extra notebook.
- Best choice: two packs, 18, with a spare notebook.
Each option is written down, compared, and checked. By laying the options side by side, the model notices something the quick answer missed. The final answer now rests on visible work instead of a single guess.
Why writing steps helps a language model
Remember how a language model works: it chooses each next token based on everything written so far. It does not have a hidden scratchpad in its head. Anything it has not written down, it has to handle in one jump.
Writing intermediate steps changes the problem:
- Each step is a smaller jump. Going from the question to "two packs: 9 + 9 = 18, for 8 notebooks" is easy. Going straight from the question to the final answer is hard.
- Earlier steps become input for later ones. When the model writes that options 2 and 3 both cost 18, it can see both results side by side when choosing.
- Mistakes become visible. If a step is wrong, later steps can notice ("wait, that gives only 6 notebooks") and correct course.
- QuestionThe full problem
- Step, step, stepSmall, checkable moves written out
- AnswerBuilt on the written work
In short, the written steps act as the model's working memory. The more steps a problem needs, the more this helps. For a one-step question like "What is the capital of Japan?", there is little to gain.
From prompting trick to built-in skill
Chain of thought started as something you asked for in the prompt. It helped, but it had limits:
- You had to remember to ask for it.
- The model's steps were often shallow. It would "show work" without really checking anything.
- It rarely went back and tried a different approach when the first one failed.
Reasoning models build the behavior in. They are trained to produce long, careful chains of thought on their own, including habits people did not have to prompt for: checking their work, noticing contradictions, and backtracking to try another path. The next lesson explains how that training works.
Reasoning models turn a prompting trick into a trained habit.
| Criteria | Chain-of-thought prompting | Reasoning model |
|---|---|---|
| Who triggers the steps | You, in the prompt | The model, automatically |
| Depth of the steps | Often short | Often long and detailed |
| Checks and backtracking | Rare | Common |
| Where the steps appear | In the answer itself | In a separate thinking phase |
Chain-of-thought prompting
- Who triggers the steps
- You, in the prompt
- Depth of the steps
- Often short
- Checks and backtracking
- Rare
- Where the steps appear
- In the answer itself
Reasoning model
- Who triggers the steps
- The model, automatically
- Depth of the steps
- Often long and detailed
- Checks and backtracking
- Common
- Where the steps appear
- In a separate thinking phase
What this means for your prompts
Because reasoning models already think step by step, some older prompting habits are less useful with them:
- "Think step by step" adds little. The model already does it.
- Scripting every step can get in the way. If you dictate a rigid method, you may stop the model from finding a better one.
- Clear goals and constraints matter more. Tell it what a correct answer must satisfy, and let it work out how to get there.
With fast models, asking for step-by-step reasoning is still a useful technique. The blog post Zero-Shot vs Few-Shot vs Chain-of-Thought Prompting covers when to use it. Lesson 4 covers prompting reasoning models in more detail.
Key Takeaways
- Chain of thought means writing out intermediate steps before the final answer.
- It helps because a language model has no hidden scratchpad: written steps become the working memory each next step builds on.
- Steps turn one big jump into many small ones and make mistakes visible and fixable.
- Reasoning models are trained to do this on their own, with deeper steps, self-checks, and backtracking.
- With reasoning models, give clear goals and constraints rather than "think step by step" or a rigid script.