Sooner or later every customer asks a Forward Deployed Engineer (FDE): "What will this cost per month?" A confident, well-reasoned estimate builds trust; a guess that's off by 10× ends projects. This guide shows how to estimate LLM application cost from first principles, with a worked example and the levers that bring it down.
Note on prices: model prices change often and differ by provider, region and model. The rates in the example below are illustrative placeholders, not real prices. Always take current rates from the provider's pricing page (for example Amazon Bedrock or Azure OpenAI in Microsoft Foundry) on the day you estimate.
What drives the bill
- Input tokens: system prompt, conversation history, retrieved chunks, tool definitions and the user's question.
- Output tokens: the answer, plus any tool-call arguments. Output is usually priced several times higher than input.
- Number of model calls per request: agents often make 3–10 calls (plan, tool calls, reflection, final answer).
- Embeddings: a one-time cost to index documents, plus re-indexing and one embedding per query.
- Infrastructure: vector database, compute for the API, logging and monitoring. Often larger than people expect at low volume.
The formula
monthly model cost =
requests per month
× calls per request
× ( input tokens per call × input price per token
+ output tokens per call × output price per token )
monthly total = model cost + embedding cost + infrastructure + buffer (20–30%)
Rough conversion: 1,000 tokens is about 750 English words. Measure your real prompts with the provider's tokenizer rather than guessing.
Worked example: policy assistant
A RAG assistant for 2,000 employees. Assumptions agreed with the customer:
| Item | Assumption |
|---|---|
| Active users per working day | 600 |
| Questions per active user per day | 4 |
| Working days per month | 22 |
| Requests per month | 600 × 4 × 22 = 52,800 |
| Model calls per request | 2 (query rewrite + answer) |
| Input tokens per call | 3,000 (system prompt 500, 5 chunks × 450, question and history 250) |
| Output tokens per call | 350 |
| Illustrative price | input X per million tokens, output 4X per million tokens |
Tokens per month:
- Input: 52,800 × 2 × 3,000 = 316.8 million tokens
- Output: 52,800 × 2 × 350 = 36.96 million tokens
Model cost = 316.8X + 36.96 × 4X = 316.8X + 147.8X ≈ 465X per month. Plug in the real X from the price page. Notice that input tokens dominate here: retrieved chunks are the biggest cost driver, which is common in RAG.
Add embedding cost (index size × embedding price, plus monthly re-indexing), the vector database, API hosting and observability, then a 20–30% buffer for growth and retries.
Agents cost more: estimate per step
For an agent, list the steps of a typical run: plan, 2–3 tool calls (each sends the growing conversation back to the model), and a final answer. Context grows with each step, so the last calls are the most expensive. Estimate a typical and a worst-case run, and cap the number of steps in code.
Levers that cut cost
| Lever | Effect |
|---|---|
| Retrieve fewer, better chunks (reranking) | Fewer input tokens, often better answers |
| Shorter system prompts and tool definitions | Saves tokens on every call |
| Route simple requests to a smaller model | Large savings when most traffic is simple |
| Prompt caching (where the provider supports it) | Cheaper repeated prefixes such as long system prompts |
| Semantic cache for repeated questions | Skips the model entirely for near-duplicate questions |
| Trim conversation history | Stops context growing without limit |
| Cap output length and agent steps | Protects against runaway costs |
| Batch processing for offline jobs | Lower rates on some platforms for non-urgent work |
Latency and cost usually improve together. See LLM latency optimisation.
Measure, then re-estimate
- Log input and output tokens per request and per step from day one.
- Build a dashboard: cost per request, per user, per feature.
- Set budget alerts in the cloud account.
- After two weeks of pilot traffic, replace your assumptions with real numbers and send the customer an updated estimate.
Monitoring setup: AI observability and LLMOps. For cloud-wide cost practice, see FinOps interview questions.
How to present the estimate
- Show assumptions in a table the customer can change.
- Give a low, expected and high range, not one number.
- Separate one-time costs (indexing, build) from monthly costs.
- State the levers you will apply if usage grows faster than planned.
- Put it in the solution design document.
Learn it hands-on
Cost estimation and optimisation are part of the enterprise projects in our FDE PRO program in Ameerpet, Hyderabad. See also the AWS Bedrock engineer roadmap.
📞 Call +91 96660 19191 · WhatsApp · Flat No. 513, 5th Floor, Nilagiri Block, Aditya Enclave, Ameerpet, Hyderabad
Frequently Asked Questions
How do you estimate LLM cost per month?
Requests per month × model calls per request × (input tokens × input price + output tokens × output price), plus embeddings, infrastructure and a 20–30% buffer.
What usually dominates RAG cost?
Input tokens, mostly the retrieved chunks sent with every call.
Why do agents cost more?
They make several model calls per request and the context grows with each step. Estimate per step and cap the number of steps.
How can I reduce LLM costs?
Retrieve fewer, better chunks; shorten prompts; route simple requests to smaller models; use prompt and semantic caching; trim history; cap output and agent steps.
Where do I get model prices?
From the provider's current pricing page on the day you estimate. Prices change often and vary by region and model.
