AI Agent Costs: Loops, Tool Calls & Context Bloat
By TokenMath Editorial & Engineering Team · Reviewed · Methodology
Autonomous agents don't make single API calls. A typical ReAct or plan-and-execute agent executes 5 to 15 sequential iterations per task. Without careful history management, compounding context sizes will decimate your unit economics.
1. The Compounding Context Mathematics
In an autonomous agent loop:
- Step 1: Model reads System Prompt (4,000 tokens) + User Goal (100 tokens) → emits Tool Call #1 (150 tokens).
- Step 2: Model reads Step 1 context (4,250 tokens) + Tool Result (1,500 tokens) = 5,750 tokens → emits Tool Call #2.
- Step 3: Model reads Step 2 context (6,000 tokens) + Tool Result #2 (2,000 tokens) = 8,000 tokens...
By Step 8, a single planning decision re-ingests over 25,000 tokens. The total tokens consumed across all 8 steps is not 25,000, but the sum of all intermediate contexts—often exceeding 120,000 tokens for a single user task!
When an agent retains full, unpruned conversational history across turns, cumulative token consumption grows quadratically with the number of steps (N):
Where (S) is static system prompt size and (\Delta) is the average delta per turn (tool call + observation). However, with proactive scratchpad compaction, sliding context windows, or persistent KV prompt caching, effective billing growth can be compressed back down to near-linear (O(N)).
2. Mitigation: Three Rules for Production Agent Architectures
- Freeze System & Tool Schemas at the Beginning of the Prompt: Placing static tool definitions at the very start allows prompt caching to cache the largest block across every subsequent agent step.
- Truncate & Summarize Intermediate Observations: Never return raw 50KB JSON responses from SQL queries or web scrapes into the agent's active context. Run a lightweight extractor model (like Gemini Flash-Lite or Claude Haiku) to condense tool outputs to 200 tokens before feeding them to the primary agent.
- Cap Maximum Loop Depth: Hardcode an architectural circuit breaker (e.g. 10 iterations max) to prevent runaway infinite loops when an agent encounters unexpected tool errors.