AI API Pricing Models: Per-Token, Provisioned Throughput & GPU TCO
When scaling AI applications from prototype to enterprise production, selecting the wrong billing model can inflate infrastructure bills by $50,000+ per month. Here is how the four primary procurement models work.
1. Pay-As-You-Go Serverless (Price Per Million Tokens)
The default entry point for all developers. You pay strictly for the exact number of input and output tokens consumed by your API keys.
- Pros: Zero upfront commitment, instant auto-scaling to thousands of requests per minute, zero idle hardware waste.
- Cons: Highest unit cost per token at steady high scale; subject to provider rate limits (RPM/TPM tiers) during peak regional demand.
2. Provisioned Throughput Units (PTU) / Reserved Capacity
Offered by Microsoft Azure OpenAI, Google Cloud Vertex AI, and AWS Bedrock. Instead of paying per token, you reserve a dedicated slice of inference compute capacity for 1-month or 1-year terms.
- Guaranteed Latency: P99 response times remain stable even when public serverless APIs experience throttling.
- Financial Break-Even: PTU reservations only save money if your system sustains >70% continuous utilization 24 hours a day. If your traffic drops to zero on weekends, serverless is cheaper.
3. Self-Hosted Dedicated GPU Clusters (vLLM / TensorRT-LLM)
Renting raw 8xH100 SXM5 or 8xB200 servers from cloud providers (Lambda Labs, RunPod, CoreWeave, AWS) and hosting open-weights models like Llama 3.3 70B, Qwen 2.5 Coder, or DeepSeek V3.
At $2.50 to $3.50 per GPU-hour, an 8xH100 node costs approximately $15,000 to $20,000 per month. If your workload processes over 500 million tokens per day, self-hosting slashes costs by up to 80% compared to proprietary frontier APIs.
Compare API vs Dedicated GPU TCO
Simulate your exact break-even token volume.