Chain of Thought in Action

Large Language Models (LLMs) can generate poetry, code websites, and answer trivia with astonishing fluency. But ask one to solve a 3-step math word problem, and it often stumbles. Why? Because most LLMs don't think -- they predict the next word based on training data.

To move beyond parroting patterns, we need to teach models to reason. That's where Chain-of-Thought (CoT) and related techniques come in -- intermediate reasoning steps, like scratch work on a notepad before the final answer.

What Do We Mean by "Thinking"?

Apples cost $2 each. If I buy 3 apples, how much do I pay?

A non-thinking LLM might blurt out 7 -- confident but wrong. A thinking model would reason:

• Each apple = $2
• 3 apples = 2 x 3 = 6
• Final answer: $6

Reasoning: Each apple is $2. For 3 apples, 2 x 3 = 6.
Final Answer: 6
plain text

Why Base Models Struggle

Base LLMs are excellent at fluency and recall. But they struggle with compositionality (combining multiple facts correctly), multi-hop reasoning (connecting dots), and math and logic (precise symbolic reasoning).

Adding Reasoning Superpowers

• Chain-of-Thought (CoT): simple linear scratchpad.
• Least-to-Most: break a problem into smaller subproblems.
• ReAct: interleave reasoning with tool use.
• Tree-of-Thoughts: explore multiple possible reasoning paths in parallel.
• Self-Consistency: generate multiple scratchpads and vote on the best.
• Reflexion: let the model critique and revise its own reasoning.

What is Chain-of-Thought?

Question: What is 3 + 4 x 10 - 4 x 3?

Reasoning: First, solve multiplications. 4 x 10 = 40, 4 x 3 = 12.
Equation becomes 3 + 40 - 12 = 31.

Final Answer: 31
plain text

The model isn't just guessing -- it's showing its work. Benefits: Accuracy (avoids shallow mistakes), Interpretability (we can see why the model reached a conclusion), Debuggability (if it fails, we can inspect the faulty step).

A Simple CoT Implementation

A coding experiment using the OpenAI API. The idea: force the assistant into a structured reasoning loop -- START (state the problem), THINK steps (reasoning), OUTPUT (final answer).

import OpenAI from "openai";

const client = new OpenAI();

async function main(){
  const SYSTEM_PROMPT = `
  You are an intelligent assistant that solves problems STEP BY STEP.
  Steps: START, THINK and OUTPUT.
  Rules:
  - Strictly follow JSON Format.
  - Always follow the sequence START, THINK, OUTPUT.
  - Perform only one step at a time.
  - Always do multiple THINK steps before OUTPUT.
  Output: { step: "START | THINK | OUTPUT", content: "string" }
  `;

  const messages = [
    { role: "system", content: SYSTEM_PROMPT },
    { role: "user", content: "What is 334 + 99 / 22 - 43 * 99" }
  ];

  while(true){
    const response = await client.chat.completions.create({
      model: "gpt-4.1-mini", messages
    });
    const parsed = JSON.parse(response.choices[0].message.content);
    messages.push({ role: "assistant", content: JSON.stringify(parsed) });
    if(parsed.step === "START"){ console.log("START:", parsed.content); continue; }
    if(parsed.step === "THINK"){ console.log("THINK:", parsed.content); continue; }
    if(parsed.step === "OUTPUT"){ console.log("ANSWER:", parsed.content); break; }
  }
}

main();
typescript

Example Run

START: The user wants me to solve 334 + 99 / 22 - 43 * 99
THINK: Division first: 99 / 22 = 4.5
THINK: Equation becomes 334 + 4.5 - 43 * 99
THINK: Multiplication: 43 * 99 = 4257
THINK: Equation becomes 334 + 4.5 - 4257
THINK: 334 + 4.5 = 338.5
THINK: 338.5 - 4257 = -3918.5
ANSWER: -3918.5
plain text

Notice how the assistant never skips ahead -- it thinks out loud until it reaches the conclusion.

The Tradeoffs

• It increases token cost (often 2-5x more).
• Models can become verbose without added accuracy.
• Sometimes rationales are plausible-sounding but wrong (post-hoc justifications).

Conclusion

Chain-of-Thought transforms LLMs from fluent guessers into step-by-step solvers. By forcing reasoning to be explicit -- whether through fine-tuning, prompting, or coding tricks like the example above -- we make models more accurate, more interpretable, and easier to debug.