The Engineer's Guide to Reasoning Tokens: o3, o3-pro & Claude Thinking
Why do reasoning models bill invisible tokens? How does chain-of-thought compute scale, and how can engineers prevent autonomous agent loops from generating \$50 API calls on a single prompt?
1. What Are Reasoning Tokens?
Traditional frontier models (such as GPT-4o or Claude 3.5 Sonnet) produce output tokens immediately upon reading the input prefix. When presented with a complex problem, they predict the next token sequentially without an internal deliberation phase.
Reasoning models—pioneered by OpenAI with the o-series (o1, o3, o3-pro, o4-mini) and Anthropic with Claude Fable 5.1 & Claude Thinking—introduce test-time compute. Before emitting the first token of the user-visible answer, the model generates an internal, private sequence of "thought tokens" to decompose the problem, double-check edge cases, and correct false assumptions.
Reasoning tokens are billed as OUTPUT tokens, even though they are hidden from the final API response text. Because output token rates are 3x to 5x higher than input rates, a query with 5,000 internal thinking tokens will incur substantial generation costs.
2. The Economics of Test-Time Compute
Consider a mathematical proof or complex refactoring prompt:
Total Cost: $0.015
Reasoning Tokens (Hidden): 4,500 tokens ($0.036)
Answer Tokens: 600 tokens ($0.0048)
Total Cost: $0.0448 (3x higher)
3. Controlling Reasoning Budgets in Production
To maintain financial predictability, modern inference APIs provide knobs to constrain thinking depth:
- OpenAI
reasoning_effort: Configure to"low","medium", or"high". Setting effort to "low" caps thinking tokens, cutting generation bills by up to 60% on moderately difficult queries. - Anthropic
thinking.budget_tokens: Set an exact integer maximum on thinking steps (e.g. 2,048 tokens). Once the budget is exhausted, the model transitions directly to drafting the final response.
4. When to Use Reasoning Models vs Fast Models
- Complex architectural coding & multi-file refactors
- Cryptographic, mathematical, or logic puzzles
- Security vulnerability audits and exploit detection
- High-stakes agent planning where mistakes cause cascade failures
- Content generation, copywriting & email drafting
- Structured data extraction & JSON schema filling
- Natural language classification & semantic routing
- Real-time voice, customer support & interactive chatbots