The Technique That Changed Everything

In 2022, Google researchers published a paper that shocked the AI community. By adding a single phrase — "Let's think step by step" — to math and logic prompts, accuracy jumped from 17.7% to 78.7% on a standard benchmark. That phrase became the foundation of chain-of-thought (CoT) prompting, and it remains one of the most powerful techniques you can use today.

But "Let's think step by step" is the beginner version. In this lesson, you'll learn the advanced approaches that get even better results.

Why Chain-of-Thought Works

Language models generate text one token at a time. When you ask a complex question and expect a direct answer, the model has to "jump" to the conclusion without working through intermediate steps. That's like asking someone to multiply 247 × 38 in their head and give you the answer instantly — they'll probably get it wrong.

Chain-of-thought gives the model scratch paper. Each reasoning step becomes context for the next step, allowing the model to build toward the answer incrementally instead of guessing.

Level 1: Zero-Shot CoT (The Simple Version)

Just add a reasoning trigger to your prompt. This works surprisingly well for straightforward problems.

WITHOUT CoT:
A store has 45 apples. They sell 60% on Monday and half of
what's left on Tuesday. How many remain?
Answer: 12 [WRONG — model often rushes to incorrect answer]

WITH CoT:
A store has 45 apples. They sell 60% on Monday and half of
what's left on Tuesday. How many remain?

Think through this step by step, showing your work at each stage.

Answer:
Step 1: Monday sales: 60% of 45 = 27 apples sold
Step 2: After Monday: 45 - 27 = 18 apples remaining
Step 3: Tuesday sales: half of 18 = 9 apples sold
Step 4: After Tuesday: 18 - 9 = 9 apples remaining
Final answer: 9 apples [CORRECT]

Trigger phrases that work:

"Think through this step by step."
"Walk me through your reasoning before giving the answer."
"Break this down into steps and solve each one."
"Show your work, then provide the final answer."

Level 2: Structured CoT (The Better Version)

Instead of letting the model decide how to reason, you define the reasoning framework. This is dramatically more reliable for business and technical problems.

STRUCTURED CoT FOR DECISION MAKING:

We need to decide between AWS Lambda and AWS ECS for our
new microservice. It handles 50K requests/day with spiky
traffic (80% of requests come in a 4-hour window).

Analyze this decision using these steps:

Step 1 - Requirements Analysis: List the key technical
requirements this traffic pattern implies.

Step 2 - Option A (Lambda): List pros and cons for this
specific use case. Include estimated monthly cost.

Step 3 - Option B (ECS): List pros and cons for this
specific use case. Include estimated monthly cost.

Step 4 - Risk Assessment: What could go wrong with each
option? What's the worst-case scenario?

Step 5 - Recommendation: Based on steps 1-4, which option
do you recommend and why? State your confidence level.

By defining the steps, you control the reasoning path. The model can't skip the cost analysis or ignore risk assessment because you've made each step explicit.

Level 3: Few-Shot CoT (The Expert Version)

Show the model an example of good reasoning, then ask it to follow the same pattern. This is the highest-accuracy approach for repeated tasks.

FEW-SHOT CoT FOR CODE REVIEW:

I'll show you how I want code reviewed, then you'll
review new code the same way.

EXAMPLE:
Code: user_input = request.args.get('query')
       db.execute(f"SELECT * FROM users WHERE name = '{user_input}'")

Review:
1. IDENTIFY the pattern: Raw user input is interpolated
   directly into a SQL query string.
2. ASSESS the risk: This is a SQL injection vulnerability.
   An attacker could input: ' OR '1'='1 to dump the
   entire users table.
3. SEVERITY: Critical — data breach, full database access.
4. FIX: Use parameterized queries:
   db.execute("SELECT * FROM users WHERE name = ?", [user_input])
5. VERIFY: The fix separates data from code, preventing
   injection regardless of input content.

Now review this code using the same 5-step process:
[paste your code here]

The example teaches the model your exact review methodology. It won't just say "this looks risky" — it will identify, assess, rate severity, fix, and verify, exactly like your example.

When Chain-of-Thought Helps Most

CoT isn't magic for every task. Here's where it shines vs. where it's overkill:

High impact (always use CoT):

Math and calculation problems
Multi-step logical reasoning
Decision-making with tradeoffs
Debugging code (trace execution flow)
Analyzing legal or financial documents
Any task where you'd show your work on paper

Low impact (skip CoT):

Simple translations or reformatting
Creative writing (can actually hurt creativity)
Straightforward factual questions
Tasks where speed matters more than accuracy

Advanced Trick: Self-Verification CoT

Add a verification step at the end where the model checks its own work. This catches errors that slip through initial reasoning.

Solve this problem step by step.

After reaching your answer, verify it by:
1. Working backward from your answer to check it
   produces the original conditions
2. Checking each step for arithmetic or logical errors
3. Considering if the answer passes a sanity check
   (is it a reasonable number? does it make sense?)

If you find an error during verification, correct it
and show the corrected solution.

This self-verification step catches roughly 15-20% of errors that the initial chain-of-thought misses. It's especially valuable for multi-step math and financial calculations.

The "Think" Tag for Claude

If you're using Claude, you can use extended thinking mode or explicitly ask Claude to use a thinking block. Claude will reason through the problem in a dedicated thinking space before responding, giving you the benefits of CoT without cluttering the output.

Analyze this contract for potential risks.

Use your extended thinking to reason through each clause
before giving me your final analysis. I only want to see
the final structured analysis, not the reasoning steps.

This gives you CoT accuracy with clean output — the best of both worlds.

Chain-of-Thought and Step-by-Step Reasoning

Related Resources