RAG Pipeline Cost Calculator
Model the core API and compute costs of your enterprise RAG pipeline: document chunking, vector embedding ingestion, vector search querying, and generation synthesis.
RAG Corpus & Query Architecture
Ingestion + RetrievalHow Much Does a RAG System Cost to Run?
A RAG pipeline has two cost centers. Ingestion is mostly one-time: you embed your document corpus once (plus updates). Querying is recurring: every question embeds the query, searches the vector database, and sends the retrieved chunks to an LLM. For almost every production system, the LLM generation step is 90%+ of the monthly bill, because retrieved context multiplies the input tokens of every request.
RAG Cost Formula
Worked Example
10,000 documents of 5,000 tokens, embedded with text-embedding-3-small, answering 100,000 questions a month with Gemini 3.8 Flash. Each query retrieves 5 chunks of 500 tokens and generates 400 output tokens.
| One-time ingestion (50M tokens) | $1.00 |
| Monthly LLM input (300M tokens) | $225.00 |
| Monthly LLM output (40M tokens) | $150.00 |
| Monthly generation total | $375.00 |
Vector database hosting is extra and varies by provider and index size; query embeddings add well under 1% at these volumes.
How to Reduce RAG Costs
- Retrieve fewer, better chunks. Reranking to 3 chunks instead of 8 cuts input tokens by more than half.
- Cache the static prefix. System prompts and instructions qualify for prompt caching discounts of up to 90%.
- Route easy questions to a cheaper model. See the cheapest model finder.
- Batch offline re-indexing. Batch APIs discount embedding and generation jobs by about 50%.
For the architecture behind these numbers, read the RAG cost architecture guide.