LLM Token Costs and Efficiency: A Practitioner's Guide (July 2026)

Per-token pricing is the number on the pricing page. It is not your actual cost. The gap between the two is where most teams either save or waste thousands of dollars a month. This post covers both: the raw numbers across every major provider, and the strategies that determine what you actually pay.

Companion post: API Rate Limits Compared (July 2026) covers RPM, TPM, and throughput limits across the same providers.

Per-token pricing across providers
The hidden cost multipliers
Prompt caching: how it works, what it saves
Batch APIs: the easiest 50% discount
Model routing: the architecture that pays for itself
Token efficiency: not all tokens are equal
Structured outputs: fewer tokens, same information
Self-hosting vs. API: when the math flips
Cost modeling a real workload
Further reading

All prices verified against official provider documentation as of July 10, 2026. Prices change frequently. Confirm on provider pricing pages before budgeting.

Per-token pricing across providers

Prices in USD per million tokens. Sorted cheapest to most expensive by output cost within each category.

Proprietary flagship models

Model	Input/M	Output/M	Cached Input/M	Context	Notes
Mistral Small 4	$0.20	$0.60	—	256K	Hybrid instruct + reasoning + multimodal
DeepSeek V4-Flash	$0.14	$0.28	$0.028	1M
GPT-5.5 Instant	$0.20	$1.00	—	128K	Default ChatGPT model across all tiers since May 5
GPT-5.4 Nano	$0.20	$1.25	$0.05	128K	Budget/high-throughput tier
Gemini 3.1 Flash-Lite	$0.25	$1.50	—	1M	Cost-efficient, fastest in Gemini 3 series
Grok 4.3	$1.25	$2.50	—	1M	xAI flagship; leads on non-hallucination and instruction following
DeepSeek V4-Pro	$0.435	$0.87	$0.036	1M
Claude Haiku 4.5	$1.00	$5.00	$0.10	200K	Cache write: $1.25/M
Mistral Large 3	$2.00	$6.00	—	128K
Claude Sonnet 5	$2.00	$10.00	$0.20	1M	Introductory pricing through Aug 31; then $3/$15; cache write: $2.50/M
Qwen3.7-Max	$2.50	$7.50	—	1M
Gemini 3.5 Flash	$1.50	$9.00	—	1M	GA since Google I/O 2026; default Gemini app model
GPT-5.4	$2.50	$10.00	$1.25	1M
Gemini 3.1 Pro	$2–4	$12–18	—	1M	$2/$12 under 200K; $4/$18 above
Claude Sonnet 4.6	$3.00	$15.00	$0.30	1M	Cache write: $3.75/M; previous balanced tier
GPT-5.5	$5.00	$30.00	$2.50	1M	~40% more token-efficient per task than GPT-5.4 on coding
Claude Fable 5	$5.00	$25.00	$0.50	1M	95% SWE-bench; restored globally July 1 after export-control suspension
Claude Opus 4.8	$5.00	$25.00	$0.50	1M	Cache write: $6.25/M; current Anthropic reasoning flagship
GPT-5.5 Pro	Contact sales	—	—	—	Enterprise only

Several notes on this table.

Claude Sonnet 5 launched June 30, 2026, as the new balanced tier, closing much of the gap to Opus 4.8. Its new tokenizer emits approximately 30% more tokens for the same text than the tokenizer used in Sonnet 4.6 — a meaningful consideration when migrating workloads. The $2/$10 pricing is introductory through August 31, 2026, after which it reverts to the standard $3/$15 rate. Cost comparisons against Sonnet 4.6 should account for both the tokenizer difference and the price step-up.

Claude Fable 5 launched June 9, 2026, with a 95% SWE-bench score, making it Anthropic’s strongest coding model. It was suspended June 12 under a US export-control directive, then restored globally July 1 after those controls were lifted June 30. Anthropic added a classifier on redeployment that blocks the reported jailbreak technique and reroutes flagged requests to Opus 4.8. Pricing follows Anthropic’s flagship pattern — confirm before production budgeting.

Grok 4.3 launched April 30, 2026. xAI’s published pricing of $1.25/$2.50 per million tokens is unusually favorable for a model that leads on non-hallucination rate and instruction following in third-party evaluations. For workloads where factual accuracy is the primary quality constraint, it is worth evaluating against Claude Sonnet 5 and GPT-5.4 on a per-task-cost basis.

Qwen3.7-Max pricing is now confirmed at $2.50/$7.50 per million tokens on Alibaba Cloud, correcting the provisional estimate from the previous edition.

GPT-5.6 Sol was previewed June 27, 2026, with claimed improvements in coding, science, and cybersecurity. Access is limited to trusted partners in a tiered rollout (Sol/Terra/Luna variants); broader US access has been restricted. It does not appear in this table because it is not generally available for production workloads.

Gemini 3.5 Pro was announced at Google I/O 2026 with a 2M context window and Deep Think mode but remains in limited preview as of July 10. It is not in this table for the same reason.

Claude Mythos 5 has access restored for vetted US organizations via Project Glasswing (critical-infrastructure and defensive-security use only). It is not generally available and does not appear in pricing tables.

Gemini 3.1 Pro and Gemini 3.1 Flash-Lite remain available as previous-generation options within the Gemini 3 family. Their pricing is unchanged from prior editions but omitted from this table for brevity.

Reasoning/thinking models

Model	Input/M	Output/M	Context	Notes
DeepSeek R1	$0.55	$2.19	128K	Cache hits: $0.14/M
o4-mini	$1.10	$4.40	200K	Thinking tokens: $1.10/M
o3	$2.00	$8.00	200K	Thinking tokens: $2.00/M
GPT-5.4 Thinking	$3.00	$12.00	1M	Thinking tokens billed at output rate
GPT-5.5 Thinking	$6.00	$36.00	1M	Thinking tokens billed at output rate

Thinking tokens deserve sustained attention as reasoning-capable models become more common. When a model uses extended thinking, you pay for those tokens too. A complex math problem that triggers 10,000 thinking tokens before generating a 500-token answer costs you for 10,500 tokens of output, not 500. Anthropic, OpenAI, and Google all bill thinking tokens at the same rate as standard output tokens for their respective models.

Anthropic’s budget_tokens parameter lets you cap thinking tokens per request. OpenAI’s reasoning_effort parameter (low/medium/high) on o3/o4-mini controls thinking depth. Use these to bound costs on reasoning-heavy workloads.

Claude Fable 5’s 95% SWE-bench performance likely involves extended internal reasoning, though Anthropic has not published the typical thinking-token budget for coding tasks. Benchmark scores derived with unconstrained thinking budgets may not reflect costs under production budget_tokens limits.

Open-weight models (hosted pricing)

Self-hosted = free. These are representative hosted prices from providers like Groq, Together AI, Fireworks, and Alibaba Cloud.

Model	Hosted Input/M	Hosted Output/M	Context	License
Llama 4 Scout (109B/17B active)	~$0.10	~$0.40	10M	Meta
Qwen3.6-27B	~$0.12	~$0.50	128K	Apache 2.0
Llama 4 Maverick	~$0.15	~$0.60	1M	Meta
Qwen3.6-35B-A3B	~$0.15	~$0.60	128K	Apache 2.0
Gemma 3 27B	~$0.20	~$0.20	128K	Google
Mistral Small 4	~$0.20	~$0.60	256K	Apache 2.0
DeepSeek R1 (hosted)	$0.55	$2.19	128K	MIT
Command A+ (218B MoE, 25B active)	~$0.50	~$1.50	256K	Apache 2.0
Devstral 2 (123B)	$0.40	$2.00	256K	Apache 2.0
Mistral Large 3 (675B MoE, hosted)	~$2.00	~$6.00	128K	Apache 2.0

Qwen3.5 and Qwen3 are older generations; the current open-weight Alibaba family is Qwen3.6 (Apache 2.0). Qwen3.6-35B-A3B uses a mixture-of-experts architecture with 3B active parameters despite 35B total, which lowers inference cost substantially on compatible serving infrastructure.

Command A+ launched May 20, 2026, with 25B active parameters despite 218B total, sufficient for inference on 2×H100 or a single B200. Its native citation support makes it particularly useful for RAG workloads where attribution is a requirement, not an afterthought.

Devstral 2 achieved 72.2% on SWE-bench, the strongest open-weight coding score available as of this writing. For teams with coding-heavy workloads and existing GPU infrastructure, its Apache 2.0 license and 256K context make it a practical self-hosting target — though Fable 5’s 95% SWE-bench score has shifted the proprietary ceiling considerably.

Embeddings and other modalities

Model	Price	Unit	Notes
OpenAI text-embedding-3-small	$0.02/M tokens	tokens
OpenAI text-embedding-3-large	$0.13/M tokens	tokens
Gemini Embedding 2	Free (AI Studio)	tokens	Multimodal (text, image, video, audio)
Cohere Embed v3	$0.10/M tokens	tokens	Multilingual
Whisper (OpenAI)	$0.006/min	audio minutes
Whisper (Groq, free tier)	$0.00	audio minutes	8 hrs/day limit
Gemini 3.1 Flash TTS	Contact Google	characters	Dedicated TTS model
OpenAI TTS	$15.00/M chars	characters
OpenAI TTS HD	$30.00/M chars	characters
GPT Image 2	$0.02–$0.17	per image	Varies by quality/size

The hidden cost multipliers

The pricing table above is the starting point, not the answer. Several factors multiply your actual spend beyond the per-token rate.

1. Output tokens cost 2–6x more than input tokens

Every major provider charges more for output than input. The ratio varies:

Provider/Model	Typical Output:Input Ratio
DeepSeek V4-Flash	2:1
Grok 4.3	2:1
Qwen3.7-Max	3:1
GPT-5.4	4:1
Claude Sonnet 5 (introductory)	5:1
Claude Opus 4.8	5:1
GPT-5.5	6:1

Prompt engineering that reduces output length has an outsized cost impact. A system prompt that gets the model to return 200 tokens instead of 500 tokens saves more than a system prompt that reduces input by the same 300 tokens.

2. Context length pricing tiers

Google charges different rates based on how much of the context window you use. Gemini 3.1 Pro costs $2/$12 per million tokens under 200K, but $4/$18 above — a 50% price increase for long-context workloads. Gemini 3.5 Flash’s flat pricing regardless of context depth is one practical advantage over 3.1 Pro for workloads that approach the 1M token window. Other providers charge flat rates regardless of context usage, but this could change as 1M+ context windows become standard.

3. Thinking token overhead

Extended thinking is not free. A request to Claude Opus 4.8 that triggers 20,000 thinking tokens at $25/M output adds $0.50 to that single request before the actual response tokens. For complex reasoning tasks, thinking tokens can exceed response tokens by 10–50x. This cost pattern is amplified on Fable 5 for coding workloads — its 95% SWE-bench performance probably involves substantial thinking-token use on hard instances.

4. Cache write costs

Anthropic charges a 25% premium on tokens written to cache vs. standard input. Claude Opus 4.8: $6.25/M for cache writes vs. $5.00/M for standard input. The write premium pays for itself after the second cache read ($0.50/M), so any cached prefix used 3+ times is net positive. Claude Sonnet 5 follows the same pattern at $2.50/M cache write vs. $2.00/M standard input during the introductory period. Single-use prompts should skip caching.

5. Claude Sonnet 5 tokenizer inflation

Sonnet 5’s new tokenizer emits approximately 30% more tokens for the same text than the tokenizer used in Sonnet 4.6. At introductory pricing ($2/$10 vs. $3/$15), the effective cost per character is lower than Sonnet 4.6 even accounting for the token inflation. After August 31 when pricing reverts to $3/$15, the per-character cost converges closely with Sonnet 4.6. Any cost model migrating from Sonnet 4.6 to Sonnet 5 needs to re-benchmark token counts on actual workload data, not assume parity.

6. Image and multimodal token costs

Images are converted to tokens before processing. A single high-resolution image can consume 1,000–5,000+ tokens depending on the model and resolution settings. For vision-heavy workloads — document OCR, screenshot analysis, diagram interpretation — image token costs can dominate the bill. Resize images to the minimum resolution needed before sending them. Gemini 3.5 Flash’s computer-use capability (screenshots, clicking, UI navigation) introduces a pattern where a single agentic session can accumulate dozens of screenshot tokens across a task sequence; budgeting per-session rather than per-request is more accurate for those workloads.

7. Autonomous operation and agentic loops

An agent that reads its own output history accumulates input tokens roughly quadratically over a long session. Qwen3.7-Max’s 35-hour autonomous operation demonstration illustrates the ceiling: a session of that length, even at $2.50/M input, could accumulate millions of context tokens across iterative steps. Budget token limits at the task level, not just the request level, for any agentic workload. grok-build-0.1, xAI’s agentic coding model, operates under similar dynamics — multi-step coding tasks generate context that grows with each tool call and intermediate output.

8. Access-gated models

GPT-5.4-Cyber is available only to vetted teams working on defensive security applications. GPT-5.6 Sol is in limited preview for trusted partners only. Claude Mythos 5 is restricted to vetted US organizations via Project Glasswing. None of these appear on public pricing pages, and per-token costs for them are not publicly documented. Workload planning that assumes access to any of these models requires factoring in the vetting or partnership process separately from cost modeling.

Prompt caching: how it works, what it saves

Prompt caching stores the processed representation of input tokens on the provider side so subsequent requests with the same prefix skip re-processing. The savings are substantial.

Provider comparison

Provider	Cache Read Discount	Cache Write Premium	Min Cacheable	TTL	Rate Limit Benefit
Anthropic	90% off input	25% over input	1,024 tokens	5 min (auto-extends)	Cached reads don’t count toward ITPM
OpenAI	50% off input	None (automatic)	Varies	Automatic	No rate limit benefit
Google	75% off input	None (usage-based)	4,096 tokens	Configurable	No rate limit benefit
DeepSeek	80–90% off input	None	Automatic	Varies	No rate limit benefit

Anthropic’s caching has the most generous rate limit interaction: cached read tokens are excluded from ITPM limits entirely. With 80% cache hits, a 2M ITPM limit effectively handles 10M total input tokens per minute. This is a throughput multiplier, not just a cost reduction.

Claude Sonnet 5’s cache pricing during the introductory period ($0.20/M reads, $2.50/M writes) maintains the same structural discount ratios as Sonnet 4.6 — 90% off input for reads, 25% premium for writes. The tokenizer inflation applies to cached tokens as well, so re-benchmark cache hit rates on actual prompts after migration.

When caching pays off

The break-even math for Anthropic: cache write costs 25% more than standard input. Cache reads cost 90% less. After 2 cache reads, the write premium is recovered. Anything beyond that is pure savings. For a system prompt reused across thousands of requests, the savings approach 90% on that portion of every request.

Practical caching patterns

RAG with stable context: Cache the system prompt and retrieved documents as a prefix. Each follow-up query appends to the cached prefix. Works well when the same document set is queried multiple times within the TTL window.

Multi-turn conversations: Cache the conversation history prefix. Each new turn extends the cache. Anthropic’s 5-minute auto-extending TTL means active conversations stay cached indefinitely.

Batch evaluation: Cache the evaluation rubric and few-shot examples. Run hundreds of evaluations against the cached prefix. The rubric tokens (often 2,000–5,000) are paid once at write price, then read at 90% discount for every subsequent evaluation.

Agentic task context: For long-running autonomous tasks, cache the task description, tool definitions, and accumulated working memory up to the stable prefix boundary. This is particularly relevant for Qwen3.7-Max, Claude Fable 5, and Claude Opus 4.8 workloads where session context can grow into the hundreds of thousands of tokens. For Gemini 3.5 Flash computer-use sessions, caching the stable task description and tool schema before the screenshot stream begins can meaningfully reduce costs on long UI-navigation tasks.

Batch APIs: the easiest 50% discount

OpenAI, Anthropic, and Google all offer batch processing with lower pricing and higher throughput limits.

Provider	Batch Discount	Turnaround	Queue Limits
OpenAI	50% off standard pricing	Up to 24 hours	3x–100x higher than real-time TPM
Anthropic	50% off standard pricing	Up to 24 hours	Higher than real-time
Google	50% off standard pricing	Up to 24 hours	Available for Gemini 3.1+ models

OpenAI’s batch queue limits are notably generous: GPT-5.5 gets 1.5M batch tokens at Tier 1 vs. 500K real-time TPM. At Tier 5, the batch queue reaches 15 billion tokens.

The 50% discount applies to whichever tier’s pricing is in effect at submission time. For Claude Sonnet 5, that means $1.00/$5.00 per million input/output through August 31, reverting to $1.50/$7.50 after. Batch workloads submitted before the introductory period ends lock in the lower rate for their processing window — worth scheduling accordingly for large evaluation or generation runs.

Workloads that fit batch processing: content generation pipelines, bulk classification, evaluation suites, data extraction from document archives, nightly report generation, synthetic data creation. Any workload where the output is consumed minutes or hours after generation rather than in real time.

Model routing: the architecture that pays for itself

The most impactful cost optimization is not using one model for everything. A routing layer that sends requests to different models based on complexity can cut spend 60–85%.

Cost comparison: routed vs. single-model

Assume 100,000 requests/day, average 500 output tokens per request.

Single model (Claude Sonnet 5 for everything, introductory pricing):

100K requests × 500 tokens = 50M output tokens/day
50M × $10/M = $500/day

Note: Sonnet 5’s tokenizer may produce ~30% more tokens for equivalent output compared to Sonnet 4.6. If migrating from Sonnet 4.6 workloads, the actual token count per request needs to be measured before projecting costs.

Three-tier routing (70/25/5 split):

70K requests × 500 tokens × $5/M (Haiku 4.5) = $175.00
25K requests × 500 tokens × $10/M (Sonnet 5, introductory) = $125.00
5K requests × 500 tokens × $25/M (Opus 4.8 or Fable 5) = $62.50
Total: $362.50/day (27% savings over Sonnet 5 alone; savings grow substantially after Aug 31 when Sonnet 5 reverts to $15/M output)

The classifier itself costs almost nothing. Running GPT-5.4 Nano ($0.20/M input) or Haiku 4.5 ($1.00/M input) to classify intent on a 100-token input works out to 10M input tokens across 100K daily classifications: roughly $2–10/day.

For coding-specific routing, Fable 5 is the obvious complex-tier target — its 95% SWE-bench score represents a substantial gap over Opus 4.8 on hard coding tasks, at the same $25/M output price.

Open-weight routing for cost floors

For teams with GPU infrastructure, routing the simple tier to a self-hosted model (Llama 4 Scout, Qwen3.6-35B-A3B, Command A+) eliminates the per-token cost entirely on the majority of requests. Command A+‘s Apache 2.0 license and 2×H100 inference footprint (or single B200) make it a practical target for the medium tier as well, at near-zero marginal cost above infrastructure. Devstral 2 remains the dedicated open-weight coding tier for engineering-focused routing pipelines — 72.2% SWE-bench at self-hosted inference costs is competitive with mid-tier proprietary models on code tasks, though the gap to Fable 5 is now substantial for the hardest coding problems.

Token efficiency: not all tokens are equal

The same task produces different token counts on different models. This is the most underappreciated cost variable.

GPT-5.5’s efficiency claim

OpenAI reports GPT-5.5 is ~40% more token-efficient than GPT-5.4 on coding tasks. Per-token price doubled ($5/$30 vs. $2.50/$15 for standard), but if the model uses 40% fewer tokens to produce equivalent output, the effective per-task cost is ~20% higher, not 100% higher. This claim is plausible for code generation (where a more capable model produces tighter code in fewer iterations) but unlikely to hold for simple summarization or classification tasks where output length is fixed. Evaluate token efficiency on your specific workload before assuming the headline number applies.

Claude Sonnet 5 tokenizer effects

Sonnet 5’s tokenizer emitting ~30% more tokens for equivalent text is a cost variable that cuts in both directions. It inflates input costs when sending the same prompts. It also means any per-token efficiency gain from the model’s improved intelligence competes against the baseline token inflation. Whether Sonnet 5 is cheaper per task than Sonnet 4.6 depends entirely on the ratio of intelligence gain to tokenizer inflation on a given workload — which makes benchmarking on real data more important than usual during the migration window.

Grok 4.3’s per-task cost position

At $1.25/$2.50 per million tokens with a 2:1 output:input ratio, Grok 4.3 is positioned unusually low in the cost curve for a model that leads on non-hallucination and instruction following. For workloads where factual accuracy is the binding quality constraint — knowledge-base Q&A, document summarization, factual extraction — it is worth benchmarking against Sonnet 5 and GPT-5.4 on a per-task basis rather than relying on per-token comparisons alone.

GPT-5.3 Codex and the coding model landscape

GPT-5.3 Codex remains available as a specialized coding variant. Its coding-specific optimization has historically produced better token efficiency than the general GPT-5.4 line on certain code generation tasks. However, Fable 5’s 95% SWE-bench score changes the reference point for what “strong coding performance” means. Fable 5 at $25/M output costs more than GPT-5.3 Codex on a per-token basis; whether it’s cheaper per completed task depends on how many fewer tokens it needs to solve hard problems. GPT-5.2 is being phased out; GPT-5.2 Thinking was retired April 18, 2026.

Tokenizer differences matter

Different providers use different tokenizers. The same English sentence might be 15 tokens on OpenAI’s tiktoken (cl100k_base), 18 tokens on Anthropic’s tokenizer for Sonnet 4.6, and perhaps 23 tokens on Sonnet 5’s new tokenizer. For pricing comparisons across providers, convert to a common unit (characters or words) rather than comparing raw per-token prices. For comparisons within Anthropic’s own model family, re-tokenize prompts with the Sonnet 5 tokenizer before assuming cost parity with prior models.

A rough heuristic: 1 token ≈ 4 characters ≈ 0.75 words in English. This varies by language (CJK characters often consume 1–2 tokens each) and by content type (code is tokenized differently than prose). The Sonnet 5 tokenizer shifts this ratio; expect closer to 1 token ≈ 3 characters in English on that model.

Prompt engineering for token efficiency

Small changes to prompts can produce large token savings at scale:

“Respond in JSON” vs. natural language: structured responses are 30–60% shorter.
“Be concise” in the system prompt: models tend to over-explain. An explicit brevity instruction reduces output tokens by 20–40% on average.
Few-shot examples: Adding 2–3 examples of desired output length calibrates the model’s verbosity. This costs a few hundred extra input tokens but saves thousands of output tokens across the batch.
Max tokens parameter: Set a hard cap. Prevents runaway generation on edge cases where the model would otherwise produce 10x the expected output.

Structured outputs: fewer tokens, same information

JSON mode and structured outputs (GA on OpenAI, Anthropic, and Google) constrain the model to return data in a predefined schema. This is both a reliability and a cost optimization.

A sentiment classification task:

Unstructured response (typical): “The sentiment of this review is positive. The customer expresses satisfaction with the product quality and delivery speed, though they note minor concerns about packaging.” (~35 tokens)

Structured response: {"sentiment": "positive", "confidence": 0.92} (~12 tokens)

That is a 65% reduction in output tokens per request. At $10/M output (Sonnet 5, introductory), across 1M daily classifications, the savings are: (35−12) × 1M / 1M × $10 = $230/day. At $15/M post-introductory: $345/day.

For any pipeline where the downstream consumer is code rather than a human reading prose, structured outputs should be the default.

Self-hosting vs. API: when the math flips

Open-weight models (Llama 4, Qwen3.6, Mistral, DeepSeek, Command A+, Devstral 2, GPT-OSS) are free to run. The cost is infrastructure.

Break-even estimation

A single H100 (80GB) rents for roughly $2–3/hour on cloud providers. It can serve a 70B model (quantized to 4-bit) at approximately 30–50 tokens/second.

Monthly H100 cost: ~$2,000 Monthly tokens at 40 tok/s sustained: ~100B tokens

Equivalent API cost for 100B output tokens:

Claude Haiku 4.5: $500,000
Claude Sonnet 5 (introductory): $1,000,000
GPT-5.4 Nano: $125,000
DeepSeek V4-Flash: $28,000
Grok 4.3: $250,000

Self-hosting a 70B open model at $2,000/month breaks even against DeepSeek V4-Flash at roughly 7B output tokens/month. Against Haiku 4.5, it breaks even at under 500M tokens/month.

Command A+ changes the calculus somewhat. At 218B total parameters but only 25B active, it runs on 2×H100 (~$4,000/month) while delivering quality closer to mid-tier proprietary models than typical 70B open-weights. The B200 option (single card, lower power draw) reduces that infrastructure cost further for teams with access to newer hardware. For teams that can keep those GPUs near-saturated, the economics are compelling.

Devstral 2 occupies a narrower but useful niche: a 72B coding-focused model with Apache 2.0 licensing and 256K context that runs on a single H100. For teams with coding-heavy workloads, its SWE-bench performance at open-weight inference costs remains difficult to match with API alternatives at comparable volume — though teams that need Fable 5-class performance on hard coding problems will find no open-weight substitute yet.

When APIs still win

Low volume: Under ~5B tokens/month, API costs are lower than maintaining infrastructure.
Burst traffic: APIs absorb spikes. Self-hosted GPUs sit idle during troughs and can’t handle peaks without over-provisioning.
Frontier quality: No open model approaches Claude Fable 5 or GPT-5.5 on complex coding, or Claude Opus 4.8 on complex reasoning. Tasks that require the best available performance leave no self-hosted alternative.
Compliance and support: Enterprise API tiers include SLAs, BAAs (HIPAA), data residency controls, and zero-retention options that would require additional investment to replicate on self-hosted infrastructure.

When self-hosting wins

High sustained volume: Above 10B+ tokens/month, owned or reserved GPU capacity typically beats API pricing.
Data sovereignty: Air-gapped or on-premise requirements.
Fine-tuning: Self-hosted models can be fine-tuned on proprietary data. API-based fine-tuning is available from some providers but with less control.
Latency control: No network round-trip. Co-located inference can achieve sub-10ms TTFT.

Cost modeling a real workload

Pricing tables are abstractions. A concrete example of modeling costs for a production RAG pipeline follows.

Scenario: customer support agent

50,000 queries/day
Each query: 200 tokens input (user question), 2,000 tokens retrieved context, 300 tokens system prompt, 400 tokens output
System prompt + retrieval context is 80% cacheable (same FAQ docs across queries)

Note: These token counts assume the Sonnet 4.6 tokenizer. If running this workload on Sonnet 5, measure actual token counts — the ~30% tokenizer inflation means 2,500 input tokens on Sonnet 4.6 may become ~3,250 on Sonnet 5 for the same text.

Option A: Claude Sonnet 5 (introductory), no optimization

Component	Tokens/request	Daily tokens	Cost/M	Daily cost
Input (system + context + query)	2,500	125M	$2.00	$250.00
Output	400	20M	$10.00	$200.00
Total				$450.00/day

Monthly (through Aug 31): ~$13,500. After introductory period at $3/$15: ~$20,250/month.

Option B: Claude Sonnet 5, with prompt caching

Component	Tokens/request	Daily tokens	Cost/M	Daily cost
Cache write (system + context, first request/5min)	2,300	~0.7M	$2.50	$1.75
Cache read (system + context, subsequent)	2,300	114.3M	$0.20	$22.86
Uncached input (user query)	200	10M	$2.00	$20.00
Output	400	20M	$10.00	$200.00
Total				$244.61/day

Monthly (through Aug 31): ~$7,338. 46% savings from caching alone. After introductory period, the cached read rate rises proportionally — re-run this model against the post-August pricing before committing to long-term infrastructure decisions based on the introductory numbers.

Option C: Routed, with caching

Route 70% of queries (simple FAQ-type) to Haiku 4.5, 30% (complex) to Sonnet 5. Both with caching.

Component	Daily cost
Haiku tier (35K queries, cached)	~$52.50
Sonnet tier (15K queries, cached)	~$78.00
Classifier (50K queries, Haiku)	~$0.50
Total	~$131.00/day

Monthly (through Aug 31): ~$3,930. 71% savings vs. Option A.

Option D: Add batch processing for non-real-time queries

If 20% of queries (overnight analytics, report generation) can tolerate 24-hour turnaround:

Component	Daily cost
Real-time routed (40K queries)	~$104.80
Batch (10K queries, 50% discount)	~$13.10
Total	~$117.90/day

Monthly (through Aug 31): ~$3,537. 73% savings vs. Option A.

The progression from $13,500/month to $3,537/month uses no different models, no degradation in output quality, and no self-hosting. It is entirely a function of caching, routing, and batching — infrastructure decisions, not model decisions. After the Sonnet 5 introductory period ends, the absolute numbers shift upward but the relative savings from each optimization layer remain roughly constant.

Quick reference: cost per 1M output tokens

Sorted cheapest to most expensive. Input costs excluded.

Model	Output $/M	Type
Mistral Nemo 12B	$0.06	Proprietary
DeepSeek V4-Flash	$0.28	Proprietary
Llama 4 Scout (hosted)	~$0.40	Open-weight
DeepSeek V4-Pro	$0.87	Proprietary
GPT-5.5 Instant	~$1.00	Proprietary
GPT-5.4 Nano	$1.25	Proprietary
Gemini 3.1 Flash-Lite	$1.50	Proprietary
DeepSeek R1	$2.19	Open-weight
Grok 4.3	$2.50	Proprietary
Qwen3.7-Max	$7.50	Proprietary
Gemini 3.5 Flash	$9.00	Proprietary
Claude Sonnet 5	$10.00	Proprietary (introductory through Aug 31; $15 after)
GPT-5.4	$10.00	Proprietary
GPT-5.4 Thinking	$12.00	Proprietary
Gemini 3.1 Pro	$12–18	Proprietary
Claude Sonnet 4.6	$15.00	Proprietary
Claude Fable 5	$25.00	Proprietary
Claude Opus 4.7	$25.00	Proprietary
Claude Opus 4.8	$25.00	Proprietary
GPT-5.5	$30.00	Proprietary
GPT-5.5 Thinking	$36.00	Proprietary