LLM API Pricing Explained: Input, Output & Reasoning Tokens
By TokenMath Editorial & Engineering Team · Reviewed · Methodology
API pricing pages display rates "per 1 million tokens", but why is output generation so much more expensive than input ingestion? And why do reasoning models like o3 generate unexpectedly high invoices?
1. Why Output Tokens Are Often More Expensive Than Input Tokens
The asymmetry between input and output pricing reflects fundamental GPU hardware mechanics:
- Input Prefill is Parallel: When you provide a 4,000-token prompt, modern GPUs process all 4,000 tokens concurrently in parallel matrix multiplications across thousands of Tensor Cores. Prefill is compute-bound and highly hardware-efficient.
- Output Generation is Sequential: Generation is autoregressive. To emit token #101, the model must wait for token #100. Each output token requires loading all model weights from GPU memory to the compute units, making generation strictly memory-bandwidth bound.
The output-to-input price ratio varies by architecture and provider strategy:
2. The Reasoning Token Trap
Reasoning models (like OpenAI o3, o3-pro, and Claude extended thinking) do not just generate an answer. They generate an invisible chain-of-thought deliberation log that can span 1,000 to 16,000 tokens before emitting the first user-facing word.
These hidden reasoning tokens are billed as output tokens at full output rates ($8.00/1M on o3). A single query with 5,000 hidden reasoning tokens costs $0.04 just for the internal thinking phase!
3. The Blended 3:1 Industry Standard
To compare models fairly, engineers use a blended cost metric assuming an average prompt-to-completion ratio of 3:1 (3 input tokens for every 1 output token):
For Claude Sonnet 5 ($2.00 in / $10.00 out), the blended rate is $4.00 / 1M tokens.
Run npx @tokenmath/tokenmath calculate --model claude-sonnet-5 --input 15000 --output 2500 or explore the Terminal CLI documentation.