Long-Context Pricing: Tier Multipliers & Budget Traps
When working with million-token models, developers often assume the headline per-token rate applies linearly across all prompt sizes. On tiered models, crossing a single threshold doubles the entire request cost.
1. The 200k Token Step Function: Gemini 3.1 Pro
Google Cloud’s flagship Gemini 3.1 Pro model features a 1,048,576 token context window, but implements a strict two-tier pricing step:
Crucially, when a prompt reaches 200,001 tokens, the entire request is billed at Tier 2 rates, not just the single token above 200k. A prompt of 199,000 tokens costs $0.25; adding just 2,000 extra tokens increases the bill to $0.50!
2. Why Providers Charge Context Surcharges
As detailed in our attention complexity analysis, maintaining KV cache tensors for sequences beyond 200k tokens consumes significant VRAM allocations on 8xH100 clusters. To prevent cluster memory exhaustion and discourage unnecessary context dumping, providers apply pricing step functions to disincentivize inefficient prompting.
3. Architectural Best Practices
- Aggressive Context Truncation: In multi-turn chat applications, prune tool returns and history tokens so the prompt stays comfortably below the 200k boundary (e.g. 185k limit).
- Two-Stage RAG Filtering: Use dense vector retrieval and neural reranking to pass only the top-10 most relevant document chunks rather than dumping raw 500-page PDFs into the context window.
Model Gemini 3.1 Pro Long-Context Costs
Toggle above and below 200,000 tokens in the Cost Estimator.