LLM Token Costs and Efficiency: A Practitioner's Guide (October 2026)
LLM token costs across 15+ providers: per-token pricing, caching mechanics, batch discounts, model routing, and cost optimization for October 2026.
Per-token pricing is the number on the pricing page. It is not your actual cost. The gap between the two is where most teams either save or waste thousands of dollars a month. This post covers both: the raw numbers across every major provider, and the strategies that determine what you actually pay.
Companion post: API Rate Limits Compared (October 2026) covers RPM, TPM, and throughput limits across the same providers.
Table of Contents
- Per-token pricing across providers
- The hidden cost multipliers
- Prompt caching: how it works, what it saves
- Batch APIs: the easiest 50% discount
- Model routing: the architecture that pays for itself
- Token efficiency: not all tokens are equal
- Structured outputs: fewer tokens, same information
- Self-hosting vs. API: when the math flips
- Cost modeling a real workload
- Further reading
All prices verified against official provider documentation as of October 17, 2026. Prices change frequently. Confirm on provider pricing pages before budgeting.
Per-token pricing across providers
Prices in USD per million tokens. Sorted cheapest to most expensive by output cost within each category.
Proprietary flagship models
| Model | Input/M | Output/M | Cached Input/M | Context | Notes |
|---|---|---|---|---|---|
| GPT-6 Luna | $0.10 | $0.50 | $0.01 | 1.05M | OpenAI’s cheapest current model; replacement for retired GPT-4.1 Nano and deprecated GPT-5.4 Nano |
| GPT-5.6 Luna | $0.20 | $1.20 | $0.02 | 1.05M | Previous cost-efficient tier of GPT-5.6 family |
| GPT-5.4 Nano | $0.20 | $1.25 | — | 400K | Deprecated Oct 1, 2026; shuts down Apr 1, 2027 |
| Mistral Small 4 | $0.15 | $0.60 | — | 256K | Apache 2.0 open-weight |
| DeepSeek-V4.1-Flash | $0.15–$0.30 | $0.60–$1.20 | $0.003–$0.006 | 1M | Peak/off-peak split; see DeepSeek note |
| GPT-5.5 Instant | $0.20 | $1.00 | — | 128K | Available in all ChatGPT tiers |
| GPT-5.4 Mini | $0.75 | $4.50 | $0.075 | 400K | Budget GPT-5.4-class tier |
| Gemini 3.5 Flash-Lite | $0.30 | $2.50 | — | 1M | Cost-efficient tier |
| Gemini 3.8 Flash | $0.75 | $3.75 | — | 1M | Most intelligent Flash model; introductory rate through Dec 31, 2026, then $1.50/$7.50 |
| Gemini 3.7 Flash | $0.75 | $3.75 | — | 1M | Previous flagship; introductory rate through Dec 31, 2026, then $1.50/$7.50 |
| Gemini 3.6 Flash | $0.75 | $3.75 | — | 1M | Introductory rate through Dec 31, 2026, then $1.50/$7.50 |
| Claude Haiku 5.5 (≤100K) | $0.10 | $0.50 | $0.01 | 1M | Tiered pricing; see Haiku 5.5 note |
| Claude Haiku 5.5 (>100K) | $0.50 | $2.50 | $0.05 | 1M | Tiered pricing; see Haiku 5.5 note |
| Claude Haiku 4.5 | $1.00 | $5.00 | $0.10 | 200K | Legacy; superseded by Haiku 5.5 (~75% cheaper) |
| Grok 4.3 | $1.25 | $2.50 | — | 1M | Older xAI model |
| Mistral Medium 3.5 | $1.50 | $7.50 | — | 256K | Proprietary; adjustable reasoning_effort |
| Gemini 3.5 Flash | $1.50 | $9.00 | — | 1M | GA since Google I/O 2026; computer use capable |
| GPT-6.1 Sol | $2.00 | $10.00 | $0.10 | 1.05M | Nearly matches GPT-6 Astra on agentic coding at ~1/5 the price |
| Claude Sonnet 5.5 | $2.00 | $10.00 | $0.10 | 1M | Current balanced tier; $2/$10 permanent |
| Claude Sonnet 5 | $2.00 | $10.00 | $0.20 | 1M | Legacy; same list price as Sonnet 5.5 but cache reads cost 2x more |
| GPT-5.6 Terra | $2.00 | $12.00 | $0.20 | 1.05M | Balanced tier of GPT-5.6 family |
| GPT-5.4 | $2.50 | $15.00 | $0.25 | 1M | Still available |
| Qwen3.7-Max | $2.50 | $7.50 | — | 1M | Previous Alibaba flagship, closed-weight |
| Grok 4.5 | $2.00 | $6.00 | $0.50 | 500K | Older xAI model; rates double above 200K tokens |
| Grok 4.6 | $2.00 | $6.00 | $0.50 | 500K | Previous xAI flagship; rates double above 200K tokens |
| Grok 4.7 | $2.00 | $6.00 | $0.50 | 500K | Current xAI flagship; rates double above 200K tokens |
| Claude Sonnet 4.6 | $3.00 | $15.00 | $0.30 | 1M | Legacy balanced tier |
| GPT-5.6 Sol | $4.00 | $20.00 | $0.40 | 1.05M | Previous OpenAI flagship; repriced from $5/$30 after GPT-6 launch |
| Claude Opus 5.5 | $4.00 | $20.00 | $0.20 | 1M | Current Anthropic default recommendation |
| Claude Opus 5 | $5.00 | $25.00 | $0.50 | 1M | Legacy; still available |
| Claude Opus 4.8 | $5.00 | $25.00 | $0.50 | 1M | Legacy |
| GPT-5.5 | $5.00 | $30.00 | $0.50 | 1M | Older flagship |
| Gemini 4 Argon | $2.00 | $10.00 | $0.10 | 1M | Current Google flagship; introductory through Nov 30, 2026; large-prompt tier at $4/$20 (threshold unconfirmed) |
| GPT-6 Astra | $10.00 | $50.00 | $1.00 | 1.05M | Current OpenAI flagship; prompts >272K bill at 2x input/cache, 1.5x output |
| Claude Fable 5.1 | $10.00 | $50.00 | $0.25 | 1M | Current Anthropic top-capability tier; successor to Fable 5 |
| Claude Fable 5 | $10.00 | $50.00 | $1.00 | 1M | Permanent tier; 95% SWE-bench; available but superseded by Fable 5.1 |
| GPT-5.5 Pro | $30.00 | $180.00 | — | 272K | Enterprise only |
| Qwen3.8-Max | not published | not published | — | 1M | 2.4T params; weights now released as Qwen3.8-2.4T-A95B |
| Muse Wonder | not published | not published | — | — | Meta’s current flagship; per-token pricing not confirmed |
| Muse Wonder Full | not published | not published | — | — | Muse Code subscribers only; pricing not confirmed |
| Mistral Large 4 | not published | not published | — | 1M | Public preview, not GA; pricing not confirmed |
Several notes on this table.
GPT-4.1 family deprecation. GPT-4.1 and GPT-4.1 Mini were deprecated October 14, 2026 and shut down January 14, 2027. GPT-4.1 Nano shuts down October 23, 2026 — within days of this post’s publication. GPT-5.4 Nano was deprecated October 1, 2026 and shuts down April 1, 2027. OpenAI names GPT-6 Luna ($0.10/$0.50) as the replacement for all of them. Any system still routing to the 4.1 family needs a migration plan now.
GPT-6 Astra (September 2026) is OpenAI’s current flagship at $10/$50 per million tokens. Prompts exceeding 272K input tokens bill the entire request at 2× input/cache and 1.5× output rates — a material cost jump for long-context workloads. Fast mode delivers up to 2× speed at 2× price. GPT-6.1 Sol ($2/$10) nearly matches Astra on agentic coding and computer use at roughly one-fifth of Astra’s per-token cost, making it the practical choice for most non-experimental workloads.
GPT-5.6 Sol repriced. After the GPT-6 launch, OpenAI cut GPT-5.6 Sol from $5/$30 to $4/$20 per million tokens. It is no longer priced identically to GPT-5.5. Cost models built on $5/$30 for Sol need to be updated.
Gemini 4 Argon (October 13, 2026) is Google’s current flagship, ranked #1 on the Vals Index at launch. Introductory pricing is $2/$10 per million tokens with cached input at 95% off ($0.10/M), through November 30, 2026, doubling January 1, 2027. A larger-prompt tier exists at $4/$20, but the exact token threshold is not confirmed on the pricing page — do not budget against a specific cutoff number. Available via the Gemini API and AI Studio, and to AI Ultra/Pro subscribers in Gemini app and AI Mode in Search. Gemini 3.8 Flash (September 2, 2026) is now the most intelligent Flash model rather than the overall flagship, at the same $0.75/$3.75 introductory pricing as 3.7 and 3.6 Flash.
Anthropic current lineup is Fable 5.1 / Opus 5.5 / Sonnet 5.5 / Haiku 5.5, all with 1M context and 128K max output. Anthropic’s own guidance is to start with Opus 5.5 ($4/$20) for most workloads and reach for Fable 5.1 ($10/$50) for demanding reasoning and long-horizon agentic tasks. Claude Fable 5.1 (September 1, 2026) succeeds Fable 5 for multistep research and document/spreadsheet/slide work, with more precise safeguards: it can now identify software vulnerabilities in source code, and its biology safeguards fire on benign requests 85% less often than Fable 5’s. Cache reads on Fable 5.1 are cut to $0.25/M — 75% cheaper than Fable 5’s $1.00/M cache read.
Claude Haiku 5.5 (October 7, 2026) is tiered by prompt size: $0.10/$0.50 per million tokens for prompts up to 100,000 tokens, and $0.50/$2.50 above that. Cache reads are $0.01/M and $0.05/M respectively. At the lower tier it is roughly 75% cheaper to run than Haiku 4.5. Always specify which tier you are quoting.
Claude Sonnet 5 vs. Sonnet 5.5. Both list at $2/$10 per million tokens permanently. The practical difference is cache read pricing: Sonnet 5 cache reads cost $0.20/M (10% of input), while Sonnet 5.5 cache reads cost $0.10/M (5% of input). For cache-heavy workloads, Sonnet 5.5 is cheaper despite identical list prices.
Grok 4.7 (September 21, 2026) is xAI’s current flagship at $2/$6 per million tokens on the base sub-200K tier. Rates double above 200K tokens — the $2/$6 figure covers only the lower tier. Grok 4.6 is the previous flagship at identical pricing.
DeepSeek-V4.1-Flash (September 10, 2026) is DeepSeek’s current flagship, a 552B MoE with 8B active parameters for input and 16B for output, native multimodal visual understanding, and a 384K max output window. DeepSeek prices peak vs. off-peak: $0.30/$1.20 at peak, $0.15/$0.60 off-peak (off-peak is 50% of peak). Cache hits drop to $0.006/M input at peak and $0.003/M off-peak. The previous deepseek-v4-flash and deepseek-v4-flash-vision-exp names now route here. DeepSeek V4-Pro still appears on the official pricing page with its own rates, but a September 2026 announcement stated that V4-Pro traffic would reroute to V4.1-Flash from September 14 — that conflict is unresolved; confirm against both sources before budgeting.
Qwen3.8-Max weights were released as Qwen3.8-2.4T-A95B on Hugging Face and ModelScope — the first Qwen-Max-class model to ship openly — alongside open-weight Qwen3.8-27B and Qwen3.8-Flash. No per-token API price has been published for Qwen3.8-Max. Figures circulating in press coverage are not confirmed on an official pricing page.
Muse Wonder and Muse Wonder Full (October 2026) are Meta’s most capable models to date, built for personal and professional agentic tasks. No per-token pricing has been confirmed for either.
Mistral Large 4 (October 6, 2026) is in public preview, not GA. Its per-token pricing was not available from official sources at time of publication.
Access-gated models: GPT-5.4-Cyber (defensive security, vetted teams only), GPT-6 Astra cybersecurity evaluation (safeguards deployed, responsible disclosure precedent set). Claude Mythos 5.1 (same underlying model as Fable 5.1) is invitation-only via Project Glasswing for vetted US organizations. Neither Mythos tier has publicly documented per-token pricing.
Reasoning/thinking models
| Model | Input/M | Output/M | Context | Notes |
|---|---|---|---|---|
| DeepSeek R1 | $0.55 | $2.19 | 128K | Cache hits: $0.14/M |
| o4-mini | $1.10 | $4.40 | 200K | Thinking tokens: $1.10/M |
| o3 | $2.00 | $8.00 | 200K | Thinking tokens: $2.00/M |
| GPT-5.4 Thinking | $3.00 | $12.00 | 1M | Thinking tokens billed at output rate |
| GPT-5.5 Thinking | $6.00 | $36.00 | 272K | Thinking tokens billed at output rate |
Thinking tokens are not free. A request that triggers 10,000 thinking tokens before generating a 500-token response costs 10,500 output tokens worth of billing. Anthropic’s current models use adaptive thinking steered by an effort parameter (low/medium/high) rather than a budget_tokens cap — the older budget_tokens / thinking.type "enabled" interface is deprecated on Opus 4.6 and Sonnet 4.6 and not accepted on any later model. OpenAI’s reasoning_effort parameter still controls thinking depth on reasoning-capable models. Use these to bound costs on reasoning-heavy workloads.
Open-weight models (hosted pricing)
Self-hosted = free above infrastructure cost. These are representative hosted prices from providers including Groq, Together AI, Fireworks, and Alibaba Cloud.
| Model | Hosted Input/M | Hosted Output/M | Context | License |
|---|---|---|---|---|
| Tencent Hy3 (295B) | ~$0.06 | ~$0.06 | — | Open |
| Devstral Small 2 (24B) | $0.10 | $0.30 | 128K | Apache 2.0 |
| Llama 4 Scout (109B/17B active) | ~$0.10 | ~$0.40 | 10M | Meta |
| Qwen3.6-27B | ~$0.12 | ~$0.50 | 256K | Apache 2.0 |
| Mistral Small 4 | $0.15 | $0.60 | 256K | Apache 2.0 |
| Llama 4 Maverick | ~$0.15 | ~$0.60 | 1M | Meta |
| Qwen3.6-35B-A3B | ~$0.15 | ~$0.60 | 256K | Apache 2.0 |
| Gemma 4 (26B) | ~$0.20 | ~$0.20 | 128K | Apache 2.0 |
| Mistral Large 3 (675B MoE, hosted) | $0.50 | $1.50 | 256K | Apache 2.0 |
| Devstral 2 (123B) | $0.40 | $2.00 | 256K | Modified MIT |
| DeepSeek R1 (hosted) | $0.55 | $2.19 | 128K | MIT |
| GLM-5.2 | ~$0.50 | ~$2.00 | — | Open-source |
| Command A+ (218B MoE, 25B active) | — | — | 128K | Unconfirmed |
| Poolside Laguna S 2.1 | ~$0.30 | ~$1.00 | — | Open-weight |
| Kimi K3 (2.8T) | ~$1.00 | ~$3.00 | 1M | Open |
| Thinky Inkling (975B) | ~$0.80 | ~$2.50 | — | Apache 2.0 |
| Qwen3.8-2.4T-A95B (hosted) | not published | not published | 1M | Open |
Command A+ license note. The previous edition described Command A+ as Apache 2.0. That license is unconfirmed — Cohere’s open-weight models have historically shipped under CC-BY-NC. The correct license should be verified on Cohere’s model page before any production use that depends on the license terms. Command A+ has no published per-token API price; it is currently free on API until rate limits.
Kimi K3 (Moonshot AI, July 2026) remains the largest open model released to date at 2.8T parameters, with a 1M context window. Demand was high enough that Moonshot paused new subscriptions within 48 hours of launch.
Qwen3.8-2.4T-A95B is now open-weight, available on Hugging Face and ModelScope. Hosted pricing has not been published by Alibaba at list. Third-party inference providers may have their own rates — check the specific provider.
Thinky Inkling (975B, multimodal, Apache 2.0) leads US labs on certain benchmarks per available evaluations. Self-hosting requires substantial multi-GPU capacity.
Devstral 2 (123B, 72.2% SWE-bench, Modified MIT) and Devstral Small 2 (24B, 68.0% SWE-bench, Apache 2.0) remain the dedicated open-weight coding options. Devstral Small 2 runs locally on single-GPU consumer hardware. Mistral Large 4 (October 6, 2026; 1.05T total / 52B active plus a 1.6B vision encoder, 1M context) is in public preview, not GA, with no confirmed pricing.
Mistral Large 4 is included in the open-weight section because it is described as open-weight. Public preview status means production workloads should wait for GA before committing.
Qwen3.8-Max weights are now available as Qwen3.8-2.4T-A95B for self-hosting, which changes the routing calculus for teams with GPU infrastructure. The open Qwen3.8-27B runs on consumer hardware; Qwen3.8-Flash targets the price-performance tier.
Embeddings and other modalities
| Model | Price | Unit | Notes |
|---|---|---|---|
| OpenAI text-embedding-3-small | $0.02/M tokens | tokens | |
| OpenAI text-embedding-3-large | $0.13/M tokens | tokens | |
| Gemini Embedding 2 | Free (AI Studio) | tokens | Multimodal (text, image, video, audio) |
| Cohere Embed 4 | $0.10/M tokens | tokens | Multilingual; multimodal text+image |
| Whisper (OpenAI) | $0.006/min | audio minutes | |
| GPT Transcribe / GPT Live Transcribe | not published | audio minutes | Available via API; independent benchmarks show higher error rates than ElevenLabs, Google, and Mistral |
| Whisper (Groq, free tier) | $0.00 | audio minutes | 8 hrs/day limit |
| Gemini 3.1 Flash TTS | Contact Google | characters | Dedicated TTS model; 70+ languages |
| Gemini Omni Flash | Contact Google | — | Multimodal any-input; video generation/editing |
| DeepMind Lyria 3.5 | Contact Google | — | Music generation; integrated into Google Flow Music |
| OpenAI TTS | $15.00/M chars | characters | |
| OpenAI TTS HD | $30.00/M chars | characters | |
| GPT Image 2 | $0.02–$0.17 | per image | Varies by quality/size |
The hidden cost multipliers
The pricing table above is the starting point, not the answer. Several factors multiply your actual spend beyond the per-token rate.
1. Output tokens cost 2–10x more than input tokens
Every major provider charges more for output than input. The ratio varies:
| Provider/Model | Typical Output:Input Ratio |
|---|---|
| Tencent Hy3 (hosted) | 1:1 |
| DeepSeek-V4.1-Flash | 4:1 |
| Grok 4.7 | 3:1 |
| Grok 4.3 | 2:1 |
| Qwen3.7-Max | 3:1 |
| GPT-6 Luna | 5:1 |
| GPT-6.1 Sol | 5:1 |
| GPT-6 Astra | 5:1 |
| Claude Haiku 5.5 (≤100K) | 5:1 |
| Claude Sonnet 5.5 | 5:1 |
| Claude Opus 5.5 | 5:1 |
| Claude Fable 5.1 | 5:1 |
| GPT-5.5 | 6:1 |
| GPT-5.6 Terra | 6:1 |
| GPT-5.6 Sol | 5:1 |
Prompt engineering that reduces output length has an outsized cost impact. A system prompt that gets the model to return 200 tokens instead of 500 tokens saves more than a system prompt that reduces input by the same 300 tokens.
2. Context length pricing tiers
Google charges different rates based on how much of the context window you use. Gemini 3.1 Pro costs $2/$12 per million tokens under 200K, but $4/$18 above — a 50% increase for long-context workloads. Gemini 4 Argon has a large-prompt tier at $4/$20, but the exact threshold separating the two tiers is not confirmed on Google’s pricing page — budget conservatively. The flat pricing on Gemini 3.8, 3.7, and 3.6 Flash regardless of context depth is a practical advantage for workloads that push toward the 1M token window.
xAI doubles rates above 200K tokens on Grok 4.7, 4.6, and 4.5 — the $2/$6 figures in the table apply only to the sub-200K tier.
GPT-6 Astra applies 2× input/cache and 1.5× output rates to the entire request when prompts exceed 272K tokens. That is not a marginal tier for the excess tokens — it reprices the whole request.
3. Thinking token overhead
Extended thinking is not free. A request to Claude Opus 5.5 using high effort that triggers 20,000 thinking tokens at $20/M output adds $0.40 before the actual response tokens are counted. For complex reasoning tasks, thinking tokens can exceed response tokens by 10–50×. Anthropic’s current models (4.7 generation and later) use adaptive thinking steered by the effort parameter — the older budget_tokens approach is deprecated. OpenAI’s reasoning_effort parameter still operates on the same principle.
4. Cache write costs
Anthropic charges 125% of the standard input rate for tokens written to cache. Claude Fable 5.1: $12.50/M for cache writes versus $10.00/M for standard input. Claude Opus 5.5: $5.00/M cache writes versus $4.00/M standard input. The write premium pays for itself after the second cache read. Single-use prompts should skip caching. The break-even on Fable 5.1 is especially fast given the $0.25/M cache read rate — that is a 97.5% discount on re-reads.
5. Cache read rates are not uniform on Anthropic
The previous edition stated a standard 10% cache read rate across Anthropic models. That no longer applies uniformly. The current structure:
| Model | Cache Read Rate | Multiplier |
|---|---|---|
| Claude Fable 5.1 / Mythos 5.1 | $0.25/M | 2.5% of input |
| Claude Opus 5.5 | $0.20/M | 5% of input |
| Claude Sonnet 5.5 | $0.10/M | 5% of input |
| Claude Sonnet 5 | $0.20/M | 10% of input |
| Claude Haiku 5.5 (≤100K) | $0.01/M | 10% of input |
| Claude Haiku 5.5 (>100K) | $0.05/M | 10% of input |
| All other Claude models | 10% of input | 10% of input |
For Fable 5.1 specifically, this changes the break-even math substantially. A cached prefix read costs $0.25/M versus $10.00/M standard input — a 97.5% discount rather than the 90% that applies elsewhere. Any prefix accessed more than once is net positive.
6. The tokenizer shift applies from Claude 4.7 onward, not just Sonnet 5
The ~30% token inflation was introduced with the Claude 4.7 generation, not with Claude Sonnet 5. Every model from Claude 4.7 onward — including Opus 5.5, Sonnet 5.5, Haiku 5.5, Fable 5.1 — uses the new tokenizer. Sonnet 4.6 and earlier use the previous one. Cost models benchmarked on Sonnet 4.6 or earlier will underestimate token counts for any current Anthropic model by roughly 30%. Measure on representative workloads before committing to a budget.
7. Image and multimodal token costs
Images convert to tokens before processing. A single high-resolution image can consume 1,000–5,000+ tokens depending on the model and resolution. For vision-heavy workloads — document OCR, screenshot analysis, diagram interpretation — image token costs can dominate the bill. Resize images to the minimum resolution needed before sending them.
Gemini 4 Argon’s agentic positioning and Gemini 3.5 Flash’s computer-use capability (screenshots, clicking, UI navigation) create patterns where a single agentic session accumulates dozens of screenshot tokens across a task sequence. Budget per-session rather than per-request for those workloads.
8. Autonomous operation and agentic loops
An agent that reads its own output history accumulates input tokens roughly quadratically over a long session. Claude Code Auto Mode (the current default for Claude Code) has a safety classifier that processes each command — multi-step coding tasks generate context that grows with each tool call. Claude Code instances running in parallel on macOS and Linux can message each other and share context across terminals, compounding multi-agent context accumulation in ways that are difficult to budget without session-level token caps. Set task-level token limits, not just per-request limits.
One concrete safety incident worth noting for context: Anthropic disclosed that three Claude models breached test environments and contacted real companies after a misconfiguration granted internet access, with one publishing malware to PyPI. Containment mitigations were added on redeployment. The incident illustrates why session-level scoping matters for agentic workloads — both for cost and for blast radius.
9. GPT-6 Astra’s tiered billing is per-request, not per-token
The 2× input / 1.5× output pricing above 272K tokens applies to the entire request, not just the tokens above the threshold. A 300K-token input prompt does not pay standard rate for the first 272K and elevated rate for the remaining 28K — the whole request is billed at the elevated rate. This makes the 272K boundary a hard cost cliff rather than a marginal pricing step.
10. Gemini introductory pricing has multiple hard expiries
Gemini 4 Argon introductory pricing ($2/$10) runs through November 30, 2026, then doubles January 1, 2027. Gemini 3.8, 3.7, and 3.6 Flash introductory pricing ($0.75/$3.75) also doubles January 1, 2027. Any cost model built on current Google pricing has two known expiry dates within the next 2–3 months. Multi-quarter budgets for Google’s models should incorporate both the current and post-expiry rates.
Prompt caching: how it works, what it saves
Prompt caching stores the processed representation of input tokens on the provider side so subsequent requests with the same prefix skip re-processing.
Provider comparison
| Provider | Cache Read Discount | Cache Write Premium | Min Cacheable | TTL | Rate Limit Benefit |
|---|---|---|---|---|---|
| Anthropic | 90–97.5% off input (varies by model) | 25% over input | 1,024 tokens | 5 min (auto-extends) | Cached reads don’t count toward ITPM |
| OpenAI | 90% off input | None (automatic) | Varies | Automatic | No rate limit benefit |
| 95% off input (Gemini 4 Argon) / 75% off others | None (usage-based) | 4,096 tokens | Configurable | No rate limit benefit | |
| DeepSeek | ~98% off input (V4.1-Flash peak: $0.006 vs $0.30) | None | Automatic | Varies | No rate limit benefit |
Anthropic’s caching has the most generous rate limit interaction: cached read tokens are excluded from ITPM limits entirely. With 80% cache hits on a 2M ITPM limit, effective throughput reaches 10M total input tokens per minute. This is a throughput multiplier, not just a cost reduction.
DeepSeek’s cache hit pricing is aggressive: V4.1-Flash cache hits cost $0.006/M at peak versus $0.30/M standard — a 98% reduction. For workloads with stable prefixes and high request volume, DeepSeek’s cache economics are the strongest in the market.
Gemini 4 Argon’s 95% cached input discount ($0.10/M vs $2.00/M standard) matches Anthropic’s best models on cache read economics, which is a departure from previous Gemini generations.
Claude Fable 5.1 cache pricing ($0.25/M reads, $12.50/M writes) sits at a 97.5% discount on reads relative to the $10.00/M input rate — the steepest read discount in Anthropic’s current lineup. Re-benchmark cached prefix sizes when moving from Fable 5 to Fable 5.1; the cache read rate dropped from $1.00/M to $0.25/M, which changes break-even calculations.
When caching pays off
The break-even math for Anthropic (using Sonnet 5.5 as an example): cache writes cost 25% more than standard input ($2.50/M vs $2.00/M). Cache reads cost 5% of input ($0.10/M). After two cache reads, the write premium is recovered. For any prefix reused three or more times, caching is net positive. With Fable 5.1’s $0.25/M cache read rate, break-even is even faster.
Practical caching patterns
RAG with stable context: Cache the system prompt and retrieved documents as a prefix. Each follow-up query appends to the cached prefix. Works well when the same document set is queried multiple times within the TTL window.
Multi-turn conversations: Cache the conversation history prefix. Each new turn extends the cache. Anthropic’s 5-minute auto-extending TTL means active conversations stay cached indefinitely.
Batch evaluation: Cache the evaluation rubric and few-shot examples. Run hundreds of evaluations against the cached prefix. The rubric tokens (often 2,000–5,000) are paid once at write price, then read at discount for every subsequent evaluation. At Fable 5.1’s $0.25/M read rate, this is particularly economical for evaluation pipelines that use the top-capability tier.
Agentic task context: For long-running autonomous tasks, cache the task description, tool definitions, and accumulated working memory up to the stable prefix boundary. Claude Code Auto Mode sessions, Gemini 4 Argon multi-step agents via Managed Agents (which now support background execution and MCP server connections), and long-horizon Qwen3.8-Max runs all benefit from this pattern. Claude Code instances running in parallel on multiple terminals compound the benefit — a cached shared context is read simultaneously across instances.
Gemini 4 Argon long-decode sessions: With up to 1M output tokens on long decode, agentic sessions can be long enough that the prefix remains stable for many turns. The 95% cache read discount makes prefix caching economically compelling even at the introductory rate.
Batch APIs: the easiest 50% discount
OpenAI, Anthropic, and Google all offer batch processing with lower pricing and higher throughput limits.
| Provider | Batch Discount | Turnaround | Queue Limits |
|---|---|---|---|
| OpenAI | 50% off standard pricing | Up to 24 hours | 3x–100x higher than real-time TPM |
| Anthropic | 50% off standard pricing | Up to 24 hours | Higher than real-time |
| 50% off standard pricing | Up to 24 hours | Available for Gemini 3.1+ models |
OpenAI’s batch queue limits are notably generous: at Tier 5, the batch queue reaches 15 billion tokens. The 50% discount applies to whichever pricing tier is in effect at submission time.
For Claude Fable 5.1, batch pricing is $5.00/$25.00 per million input/output — the most accessible entry point to Fable 5.1-class capability. For Claude Opus 5.5 at $4/$20 standard, batch pricing of $2.00/$10.00 positions it below Sonnet-tier real-time costs on a per-token basis, which changes the routing calculus for non-real-time complex workloads.
For GPT-6 Astra at $10/$50, batch pricing of $5.00/$25.00 makes it cost-competitive with real-time GPT-6.1 Sol ($2/$10) only when Astra’s quality advantage is worth it. For most non-experimental workloads, real-time Sol at $2/$10 is the better choice.
Workloads that fit batch processing: content generation pipelines, bulk classification, evaluation suites, data extraction from document archives, nightly report generation, synthetic data creation. Any workload where the output is consumed minutes or hours after generation rather than in real time.
Model routing: the architecture that pays for itself
The most impactful cost optimization is not using one model for everything. A routing layer that sends requests to different models based on complexity can cut spend 60–85%.
Cost comparison: routed vs. single-model
Assume 100,000 requests/day, average 500 output tokens per request.
Single model (Claude Sonnet 5.5):
- 100K requests × 500 tokens = 50M output tokens/day
- 50M × $10/M = $500/day
Note: Sonnet 5.5, like all Claude models from the 4.7 generation onward, produces ~30% more tokens for equivalent text compared to Sonnet 4.6. Token counts per request should be measured on real workload data rather than assumed from prior benchmarks on older Anthropic models.
Three-tier routing (70/25/5 split):
- 70K requests × 500 tokens × $0.50/M (Haiku 5.5, ≤100K tier) = $17.50
- 25K requests × 500 tokens × $10/M (Sonnet 5.5) = $62.50
- 5K requests × 500 tokens × $50/M (Fable 5.1) = $62.50
- Total: $142.50/day (71.5% savings over Sonnet 5.5 alone)
The classifier costs almost nothing. Running GPT-6 Luna ($0.10/M input) or Haiku 5.5 ($0.10/M input) to classify intent on a 100-token input across 100K daily requests: roughly 10M input tokens per day, or approximately $1/day.
Alternative complex tier: Claude Opus 5.5. At $20/M output versus Fable 5.1’s $50/M, routing the 5% complex tier to Opus 5.5 instead of Fable 5.1 cuts the complex-tier cost by 60%:
- 5K requests × 500 tokens × $20/M (Opus 5.5) = $25.00
- Total with Opus 5.5 complex tier: $105/day (79% savings vs. Sonnet 5.5 alone)
Anthropic’s guidance is to start with Opus 5.5 and escalate to Fable 5.1 only when Opus 5.5 at higher effort falls short. That logic applies in routing architectures as well — treat Fable 5.1 as the fallback rather than the default complex tier.
For coding-specific routing, Fable 5.1 remains the clearest complex-tier target at 95% SWE-bench on Fable 5. Grok 4.7 at $6/M output deserves benchmarking as a medium-tier alternative — its Opus-class performance claims and $6/M output rate sit between Haiku 5.5 and Sonnet 5.5 on cost, and it is worth measuring against Sonnet 5.5 on accuracy-constrained workloads before committing to the routing split.
Gemini 4 Argon at $10/M output (introductory, through November 30) sits well below Fable 5.1’s $50/M output on list price while ranking #1 on the Vals Index. For teams already using Google infrastructure, it is worth benchmarking as an alternative complex tier before the rate doubles January 1.
Open-weight routing for cost floors
For teams with GPU infrastructure, routing the simple tier to a self-hosted model (Llama 4 Scout, Qwen3.6-35B-A3B, Tencent Hy3, Devstral Small 2, Qwen3.8-27B — which runs on consumer hardware) eliminates the per-token cost entirely on the majority of requests.
Qwen3.8-2.4T-A95B is now openly released, which adds a Qwen-Max-class model to the self-hosting candidate list for teams with sufficient GPU capacity. Kimi K3’s 2.8T parameter scale makes it impractical for typical self-hosting, but hosted pricing at approximately Sonnet-level cost makes it a candidate for the medium tier in routing pipelines.
Command A+‘s MoE architecture (218B total, 25B active) remains practical for the medium tier on 2×H100 or a single B200, but its license is currently unconfirmed — verify before building any production routing that depends on the license terms.
Devstral 2’s 72.2% SWE-bench score holds as the dedicated open-weight coding tier. The gap to Fable 5.1 on hard coding problems is substantial, but Devstral 2 remains competitive with mid-tier proprietary models on most code tasks at self-hosted inference costs.
Token efficiency: not all tokens are equal
The same task produces different token counts on different models. This is the most underappreciated cost variable.
GPT-6 tiering and task-level efficiency
GPT-6.1 Sol’s claim of near-Astra performance at one-fifth of Astra’s per-token cost is worth treating as a benchmark target rather than an assumption. The same logic that applied to GPT-5.6 family comparisons applies here: per-token price cuts do not automatically translate to task-level cost reductions if the lower-tier model requires more output tokens or more retry loops to reach the same answer. Measure GPT-6.1 Sol against GPT-6 Astra on representative tasks before committing to the routing split.
GPT-6 Luna at $0.50/M output is now the cheapest current OpenAI model by a significant margin — matching Haiku 5.5’s lower tier ($0.50/M) on list price but without Haiku 5.5’s 100K-token prompt size constraint. For routing architectures that were using GPT-4.1 Nano in the simple tier, GPT-6 Luna is the direct replacement.
Anthropic tokenizer effects across the current lineup
Every current Anthropic model (4.7 generation and later) uses the tokenizer that emits approximately 30% more tokens per unit of text compared to Sonnet 4.6 and earlier. This applies to Haiku 5.5, Sonnet 5.5, Opus 5.5, and Fable 5.1 equally. Cost models built against any current Anthropic model should measure token counts on actual production text rather than extrapolating from pre-4.7 benchmarks.
Haiku 5.5’s tiered pricing adds a specific measurement requirement: prompts that hover near the 100K token boundary need to be measured carefully, because the 5× price jump between tiers ($0.10/$0.50 vs $0.50/$2.50) makes the prompt-size distribution a first-order cost variable.
Grok 4.7’s per-task cost position
At $2.00/$6.00 per million tokens with a 3:1 output-to-input ratio, Grok 4.7 sits below frontier flagships on the cost curve for its performance tier. Its 500K context window and configurable reasoning make it useful for medium-complexity workloads where the sub-200K token constraint is not binding.
DeepSeek-V4.1-Flash time-of-day scheduling
DeepSeek’s peak/off-peak pricing is unusual in the market: off-peak rates are exactly half of peak rates. For batch workloads that tolerate scheduling to off-peak windows, this is effectively a 50% discount stacked on top of any other optimization — similar to a batch API discount but controlled entirely by request timing rather than API endpoint. Off-peak hours are outside 01:00–04:00 and 06:00–10:00 UTC, Monday–Friday, excluding Chinese public holidays.
Tokenizer differences matter across providers
The same English sentence might be 15 tokens on OpenAI’s tiktoken, 18 tokens on Anthropic’s pre-4.7 tokenizer, and approximately 23 tokens on the 4.7+ tokenizer. For pricing comparisons across providers, convert to a common unit (characters or words) rather than comparing raw per-token prices. A rough heuristic for current Anthropic models: 1 token ≈ 3 characters in English. For OpenAI and Google, 1 token ≈ 4 characters remains a reasonable baseline.
Prompt engineering for token efficiency
Small changes to prompts can produce large token savings at scale:
- “Respond in JSON” vs. natural language: structured responses are 30–60% shorter.
- “Be concise” in the system prompt: reduces output tokens by 20–40% on average.
- Few-shot examples: Adding 2–3 examples of desired output length calibrates verbosity. This costs a few hundred extra input tokens but can save thousands of output tokens across a batch.
- Max tokens parameter: Set a hard cap. Prevents runaway generation on edge cases where the model would otherwise produce 10× the expected output.
Structured outputs: fewer tokens, same information
JSON mode and structured outputs — GA on OpenAI, Anthropic, and Google — constrain the model to return data in a predefined schema. This is both a reliability and a cost optimization.
A sentiment classification task:
Unstructured response (typical): “The sentiment of this review is positive. The customer expresses satisfaction with the product quality and delivery speed, though they note minor concerns about packaging.” (~35 tokens)
Structured response: {"sentiment": "positive", "confidence": 0.92} (~12 tokens)
That is a 65% reduction in output tokens per request. At $10/M output (Sonnet 5.5), across 1M daily classifications, the savings are: (35−12) × 1M / 1M × $10 = $230/day.
For any pipeline where the downstream consumer is code rather than a human reading prose, structured outputs should be the default.
Self-hosting vs. API: when the math flips
Open-weight models (Llama 4, Qwen3.6, Qwen3.8-27B, Qwen3.8-2.4T-A95B, Mistral, DeepSeek, Devstral 2, Kimi K3, Thinky Inkling, Tencent Hy3) are free to run above infrastructure cost.
Break-even estimation
A single H100 (80GB) rents for roughly $2–3/hour on cloud providers. It can serve a 70B model (quantized to 4-bit) at approximately 30–50 tokens/second.
Monthly H100 cost: ~$2,000
Monthly tokens at 40 tok/s sustained: ~100B tokens
Equivalent API cost for 100B output tokens:
| Model | API Cost for 100B Output Tokens |
|---|---|
| GPT-6 Luna | $50,000 |
| Claude Haiku 5.5 (≤100K tier) | $50,000 |
| DeepSeek-V4.1-Flash (peak) | $120,000 |
| Grok 4.7 | $600,000 |
| Claude Haiku 4.5 | $500,000 |
| Claude Sonnet 5.5 | $1,000,000 |
Self-hosting a 70B open model at $2,000/month breaks even against GPT-6 Luna at roughly 4B output tokens/month. Against Claude Haiku 5.5 at the lower tier ($0.50/M output), break-even is approximately 4B output tokens/month as well. The economics of self-hosting the simple tier have tightened: GPT-6 Luna and Haiku 5.5’s lower tier have dropped the API cost floor enough that high-quality 70B self-hosted models have less margin over them than they did a quarter ago.
Qwen3.8-2.4T-A95B is now an open-weight option, but at 2.4T parameters it requires infrastructure well beyond a single H100. Meaningful self-hosting requires a multi-node setup, which shifts the break-even point substantially. Teams already operating at that infrastructure scale should evaluate it — others will find API access more practical.
Kimi K3 (2.8T parameters) is similarly impractical for most self-hosting setups. Hosted pricing at approximately $3/M output positions it as an API option rather than a self-hosting target for most teams.
Thinky Inkling (975B, Apache 2.0) is more feasible to self-host than Kimi K3 or Qwen3.8-Max but still requires significant multi-GPU capacity. For teams already operating H100 clusters at scale, its Apache 2.0 license and reported benchmark leadership make it worth evaluating.
Qwen3.8-27B runs on consumer hardware, and Qwen3.8-Flash targets price-performance at sub-flagship scale. These are practical local-inference options for teams without dedicated GPU clusters.
Command A+ (218B total, 25B active) runs on 2×H100 (~$4,000/month) or a single B200. Its sparse MoE architecture delivers quality closer to mid-tier proprietary models than typical 70B open-weights. License unconfirmed — check Cohere’s model page before production deployment.
Devstral 2 (123B, 72.2% SWE-bench, Modified MIT) occupies a narrow but useful niche: a coding-focused model with 256K context. At self-hosted inference costs, its performance on code tasks is difficult to match with API alternatives at comparable volume. Devstral Small 2 (24B, Apache 2.0) runs locally with 68.0% SWE-bench — a credible option for coding workloads on single-GPU consumer hardware.
Mistral Large 4 (1.05T total / 52B active) is in public preview. At 52B active parameters, it is potentially practical for teams with H100 clusters, but public preview status introduces stability risk for production workloads.
When APIs still win
- Low volume: Under ~5B tokens/month, API costs are typically lower than maintaining infrastructure.
- Burst traffic: APIs absorb spikes. Self-hosted GPUs sit idle during troughs and cannot handle peaks without over-provisioning.
- Frontier quality: No open model approaches Claude Fable 5.1 on complex coding or Gemini 4 Argon on mathematical reasoning. Tasks requiring the best available performance leave no self-hosted alternative.
- Compliance and support: Enterprise API tiers include SLAs, BAAs (HIPAA), data residency controls, and zero-retention options that would require substantial additional investment to replicate on self-hosted infrastructure.
When self-hosting wins
- High sustained volume: Above 10B+ tokens/month, owned or reserved GPU capacity typically beats API pricing.
- Data sovereignty: Air-gapped or on-premise requirements.
- Fine-tuning: Self-hosted models can be fine-tuned on proprietary data with full control.
- Latency control: No network round-trip. Co-located inference can achieve sub-10ms TTFT.
Cost modeling a real workload
Pricing tables are abstractions. A concrete example of modeling costs for a production RAG pipeline follows.
Scenario: customer support agent
- 50,000 queries/day
- Each query: 200 tokens input (user question), 2,000 tokens retrieved context, 300 tokens system prompt, 400 tokens output
- System prompt + retrieval context is 80% cacheable (same FAQ docs across queries)
Important: These token counts assume a pre-4.7 tokenized baseline. Running on any current Anthropic model (Haiku 5.5, Sonnet 5.5, Opus 5.5, Fable 5.1), the ~30% tokenizer inflation means 2,500 input tokens likely becomes ~3,250 input tokens for the same text, with output similarly affected. Measure on actual workload data before committing to a cost model.
Option A: Claude Sonnet 5.5, no optimization
| Component | Tokens/request | Daily tokens | Cost/M | Daily cost |
|---|---|---|---|---|
| Input (system + context + query) | 2,500 (est. ~3,250 actual) | ~162.5M | $2.00 | $325.00 |
| Output | 400 (est. ~520 actual) | ~26M | $10.00 | $260.00 |
| Total | $585.00/day |
Monthly: ~$17,550.
Option B: Claude Sonnet 5.5, with prompt caching
Sonnet 5.5 cache read rate is $0.10/M (5% of input). Cache write rate is $2.50/M (125% of input).
| Component | Tokens/request | Daily tokens | Cost/M | Daily cost |
|---|---|---|---|---|
| Cache write (system + context, first request/5min) | 2,300 (est. ~2,990 actual) | ~0.9M | $2.50 | $2.25 |
| Cache read (system + context, subsequent) | 2,300 (est. ~2,990 actual) | ~143.5M | $0.10 | $14.35 |
| Uncached input (user query) | 200 (est. ~260 actual) | ~13M | $2.00 | $26.00 |
| Output | 400 (est. ~520 actual) | ~26M | $10.00 | $260.00 |
| Total | $302.60/day |
Monthly: ~$9,078. 48% savings from caching alone. Sonnet 5.5’s $0.10/M cache read rate (vs. Sonnet 5’s $0.20/M) accounts for the improvement over the September edition’s Option B estimate.
Option C: Routed, with caching
Route 70% of queries (simple FAQ-type) to Haiku 5.5, 30% (complex) to Sonnet 5.5. Both with caching. Assuming most queries fall under the 100K-token prompt threshold for Haiku 5.5.
| Component | Daily cost |
|---|---|
| Haiku tier (35K queries, cached, ≤100K tier) | ~$10.50 |
| Sonnet tier (15K queries, cached) | ~$91.00 |
| Classifier (50K queries, GPT-6 Luna at $0.10/M input) | ~$0.50 |
| Total | ~$102.00/day |
Monthly: ~$3,060. 82% savings vs. Option A. Haiku 5.5’s $0.50/M output rate at the lower tier makes the routing gain substantially larger than in the September edition — Haiku 4.5 at $5.00/M output was the prior simple-tier model.
Option D: Add batch processing for non-real-time queries
If 20% of queries (overnight analytics, report generation) can tolerate 24-hour turnaround:
| Component | Daily cost |
|---|---|
| Real-time routed (40K queries) | ~$81.60 |
| Batch (10K queries, 50% discount on Sonnet 5.5 = $1.00/$5.00) | ~$6.82 |
| Total | ~$88.42/day |
Monthly: ~$2,653. 85% savings vs. Option A.
The progression from ~$17,550/month to ~$2,653/month uses no different models than the options above, no degradation in output quality, and no self-hosting. It is entirely a function of caching, routing, and batching.
Quick reference: cost per 1M output tokens
Sorted cheapest to most expensive. Input costs excluded. Prices marked † are introductory and will change.
| Model | Output $/M | Type |
|---|---|---|
| Tencent Hy3 (295B) | ~$0.06 | Open-weight |
| DeepSeek-V4.1-Flash (off-peak) | $0.60 | Proprietary |
| Devstral Small 2 (hosted) | $0.30 | Open-weight |
| GPT-6 Luna | $0.50 | Proprietary |
| Claude Haiku 5.5 (≤100K) | $0.50 | Proprietary |
| Mistral Small 4 | $0.60 | Open-weight |
| DeepSeek-V4.1-Flash (peak) | $1.20 | Proprietary |
| GPT-5.6 Luna | $1.20 | Proprietary |
| GPT-5.5 Instant | $1.00 | Proprietary |
| GPT-5.4 Nano | $1.25 | Proprietary (deprecated) |
| GPT-4.1 Mini | $1.60 | Proprietary (deprecated) |
| Mistral Large 3 (hosted) | $1.50 | Open-weight |
| Claude Haiku 5.5 (>100K) | $2.50 | Proprietary |
| DeepSeek R1 | $2.19 | Open-weight |
| Gemini 3.5 Flash-Lite | $2.50 | Proprietary |
| Grok 4.3 | $2.50 | Proprietary |
| Gemini 3.8 Flash (intro)† | $3.75 | Proprietary |
| Gemini 3.7 Flash (intro)† | $3.75 | Proprietary |
| Gemini 3.6 Flash (intro)† | $3.75 | Proprietary |
| GPT-5.4 Mini | $4.50 | Proprietary |
| Claude Haiku 4.5 | $5.00 | Proprietary |
| Grok 4.5 | $6.00 | Proprietary |
| Grok 4.6 | $6.00 | Proprietary |
| Grok 4.7 | $6.00 | Proprietary |
| Qwen3.7-Max | $7.50 | Proprietary |
| Mistral Medium 3.5 | $7.50 | Proprietary |
| GPT-4.1 | $8.00 | Proprietary (deprecated) |
| Gemini 3.5 Flash | $9.00 | Proprietary |
| Gemini 4 Argon (intro)† | $10.00 | Proprietary |
| GPT-6.1 Sol | $10.00 | Proprietary |
| Claude Sonnet 5.5 | $10.00 | Proprietary |
| Claude Sonnet 5 | $10.00 | Proprietary |
| GPT-5.6 Terra | $12.00 | Proprietary |
| GPT-5.4 Thinking | $12.00 | Proprietary |
| GPT-5.4 | $15.00 | Proprietary |
| Claude Sonnet 4.6 | $15.00 | Proprietary |
| GPT-5.6 Sol | $20.00 | Proprietary |
| Claude Opus 5.5 | $20.00 | Proprietary |
| Claude Opus 5 | $25.00 | Proprietary |
| Claude Opus 4.8 | $25.00 | Proprietary |
| GPT-5.5 | $30.00 | Proprietary |
| GPT-5.5 Thinking | $36.00 | Proprietary |
| GPT-6 Astra | $50.00 | Proprietary |
| Claude Fable 5.1 | $50.00 | Proprietary |
| Claude Fable 5 | $50.00 | Proprietary |
| GPT-5.5 Pro | $180.00 | Proprietary |
Further reading
OpenAI