LLM Token Costs and Efficiency: A Practitioner's Guide (August 2026)

Per-token pricing is the number on the pricing page. It is not your actual cost. The gap between the two is where most teams either save or waste thousands of dollars a month. This post covers both: the raw numbers across every major provider, and the strategies that determine what you actually pay.

Companion post: API Rate Limits Compared (August 2026) covers RPM, TPM, and throughput limits across the same providers.

Per-token pricing across providers
The hidden cost multipliers
Prompt caching: how it works, what it saves
Batch APIs: the easiest 50% discount
Model routing: the architecture that pays for itself
Token efficiency: not all tokens are equal
Structured outputs: fewer tokens, same information
Self-hosting vs. API: when the math flips
Cost modeling a real workload
Further reading

All prices verified against official provider documentation as of August 10, 2026. Prices change frequently. Confirm on provider pricing pages before budgeting.

Per-token pricing across providers

Prices in USD per million tokens. Sorted cheapest to most expensive by output cost within each category.

Proprietary flagship models

Model	Input/M	Output/M	Cached Input/M	Context	Notes
DeepSeek V4-Flash 0731	$0.14	$0.28	$0.028	1M	Fast/efficient variant; within one point of GPT-5.6 Luna on AI Index
GPT-5.6 Luna	$0.20	$1.20	—	1.05M	Entry tier of GPT-5.6 Sol family; pricing reflects ~80% cut vs. GPT-5.5
GPT-5.5 Instant	$0.20	$1.00	—	128K	Still available; default ChatGPT model since May 5
GPT-4.1 Nano	$0.10	$0.40	$0.025	1M	Budget/high-throughput tier
Gemini 3.1 Flash-Lite	$0.25	$1.50	—	1M	Cost-efficient, fastest in Gemini 3.1 family
Gemini 3.6 Flash	$0.75	$3.75	—	1M	Google Flash tier; introductory rate through Dec 31, 2026, then $1.50/$7.50
Grok 4.5	$2.00	$6.00	—	500K	xAI current flagship; Opus-class performance per independent evaluations
Grok 4.3	$1.25	$2.50	—	1M	Previous xAI flagship, still available
DeepSeek V4-Pro	$0.435	$0.87	$0.036	1M
Claude Haiku 4.5	$1.00	$5.00	$0.10	200K	Cache write: $1.25/M
Mistral Large 3	$0.50	$1.50	—	256K	675B MoE / 41B active; Apache 2.0
Qwen3.8-Max	not published	not published	—	1M	2.4T params; announced Aug 3, 2026; weights release scheduled
Qwen3.7-Max	$2.50	$7.50	—	1M	Previous Alibaba flagship, still available
Claude Sonnet 5	$3.00	$15.00	$0.30	1M	Standard rate; $2/$10 introductory pricing runs through Aug 31, 2026; cache write: $3.75/M
Gemini 3.5 Flash	$1.50	$9.00	—	1M	GA since Google I/O 2026; previous default Gemini app model
GPT-5.6 Terra	$2.00	$12.00	—	1.05M	Mid tier of GPT-5.6 Sol family
GPT-5.4	$2.50	$15.00	$0.25	1M
Gemini 3.1 Pro	$2–4	$12–18	—	1M	$2/$12 under 200K; $4/$18 above
GPT-5.5	$5.00	$30.00	$0.50	1M	~40% more token-efficient per task than GPT-5.4 on coding
GPT-5.6 Sol	$5.00	$30.00	—	1.05M	Top tier; broadly launching July 9–11; now default in Microsoft 365 Copilot
Claude Sonnet 4.6	$3.00	$15.00	$0.30	1M	Previous balanced tier, still available; cache write: $3.75/M
Claude Fable 5	$10.00	$50.00	$1.00	1M	Permanent tier confirmed July 18; restored globally July 1; cache write: $12.50/M
Claude Opus 4.8	$5.00	$25.00	$0.50	1M	Previous Anthropic flagship; cache write: $6.25/M
Claude Opus 5	$5.00	$25.00	$0.50	1M	Current Anthropic flagship; 50% lower task cost than Fable 5 on knowledge tasks; cache write: $6.25/M
Muse Spark 1.1	—	$4.25	—	—	Meta; $0.26/task; 71.3 coding score; hallucination rate reduced 73%→38%
GPT-5.5 Pro	Contact sales	—	—	—	Enterprise only

Several notes on this table.

GPT-5.6 Sol launched broadly July 9–11, 2026, after previewing June 27. OpenAI has cut pricing up to 80% across the GPT-5.6 family relative to GPT-5.5 — approximately a 13x cost reduction over four months, attributed to recursive self-improvement and distillation. The Sol/Terra/Luna variant structure creates three tiers within the family. GPT-5.6 is now the default model in Microsoft 365 Copilot. Note that the “up to 80%” figure applies across the family, not to the flagship: Sol lists at the same $5/$30 as GPT-5.5, and the savings only appear at Terra and Luna. See the GPT-5.6 pricing section below before budgeting from the headline number.

Claude Opus 5 launched July 2026 as Anthropic’s new reasoning flagship. It matches Fable 5-level coding performance while costing approximately 50% less per task on knowledge-intensive workloads, and scored 30.2% on ARC-AGI-3 — nearly four times the previous record on that benchmark. On redeployment after the Fable 5 export-control suspension, Anthropic reroutes flagged Fable 5 requests to Opus 5 rather than Opus 4.8. For most complex workloads that previously warranted Opus 4.8, Opus 5 is now the primary target.

Claude Sonnet 5 introductory pricing ($2/$10 per million input/output) runs through August 31, 2026. Standard pricing of $3/$15 takes effect September 1. Any cost model built on the introductory numbers should be recalculated before then. The ~30% tokenizer inflation relative to Sonnet 4.6 applies either way, making the effective per-character cost of Sonnet 5 at standard pricing meaningfully higher than the list-price comparison with Sonnet 4.6 implies.

Claude Fable 5 was confirmed as a permanent tier on July 18, 2026. It is available on Claude Platform, Claude.ai, Claude Code, and Cowork (AWS/Google Cloud/Microsoft Foundry). Its 95% SWE-bench score remains the highest published figure for any generally available model.

Grok 4.5 launched approximately July 9, 2026, with independent evaluations placing it at Opus-class performance. Pricing is $2.00/$6.00 per million tokens on the base sub-200K tier — xAI doubles rates above 200K tokens, so confirm against x.ai/api before budgeting long-context workloads.

Qwen3.8-Max was announced August 3, 2026, with 2.4T parameters and a 1M context window. It ranks 5th on Text Arena and 2nd on Vision Arena per Alibaba’s published figures. Weights are scheduled for release. Alibaba has not published per-token pricing for it, and the figures circulating in press coverage do not appear on an official page — do not assume it inherits Qwen3.7-Max’s $2.50/$7.50. Check Alibaba Cloud Model Studio for your account’s actual rate.

Gemini 3.6 Flash launched July 2026 as Google’s Flash flagship. The Gemini API Managed Agents service supports it with hooks for production agentic systems and background execution with MCP server connections. Its $0.75/$3.75 rate is introductory and runs through December 31, 2026, after which it doubles to $1.50/$7.50 — a cost model built on the current number has a known expiry date. Google has since shipped Gemini 3.7 Flash on the same introductory pricing, so new work should target 3.7 rather than 3.6.

Gemini 3.5 Flash-Lite and Gemini 3.5 Flash Cyber (cybersecurity domain) also launched July 2026. Gemini 3.5 Flash remains available as the previous generation. Gemini 3.5 Pro remains in limited preview — not GA, not in this table.

GPT-5.6 Sol replaced GPT-5.3 Codex as the primary coding reference point. GPT-5.3 Codex has been consolidated into ChatGPT’s unified superapp interface rather than remaining a distinct API model. GPT-5.2 is being phased out; GPT-5.2 Thinking was retired April 18, 2026.

Muse Spark 1.1 (Meta, July 2026) is included for completeness. At $0.26/task and $4.25/M output tokens, it is positioned as a budget coding option with a 71.3 benchmark score that beats GLM-5.2. The hallucination rate reduction from 73% to 38% versus the prior version is meaningful but 38% remains high for production fact-sensitive workloads.

Access-gated models: GPT-5.4-Cyber (defensive security, vetted teams only), GPT-5.6 Sol in restricted US access beyond the initial rollout, Claude Mythos 5 (Project Glasswing, vetted US organizations for critical infrastructure and defensive security). None have publicly documented per-token pricing.

Reasoning/thinking models

Model	Input/M	Output/M	Context	Notes
DeepSeek R1	$0.55	$2.19	128K	Cache hits: $0.14/M
o4-mini	$1.10	$4.40	200K	Thinking tokens: $1.10/M
o3	$2.00	$8.00	200K	Thinking tokens: $2.00/M
GPT-5.4 Thinking	$3.00	$12.00	1M	Thinking tokens billed at output rate
GPT-5.5 Thinking	$6.00	$36.00	1M	Thinking tokens billed at output rate

Thinking tokens deserve sustained attention as reasoning-capable models become more common. When a model uses extended thinking, you pay for those tokens too. A complex math problem that triggers 10,000 thinking tokens before generating a 500-token answer costs you 10,500 output tokens, not 500. Anthropic, OpenAI, and Google all bill thinking tokens at the same rate as standard output tokens for their respective models.

Anthropic’s budget_tokens parameter lets you cap thinking tokens per request. OpenAI’s reasoning_effort parameter (low/medium/high) controls thinking depth on its reasoning-capable models. Use these to bound costs on reasoning-heavy workloads.

Claude Opus 5’s ARC-AGI-3 score of 30.2% — nearly four times the previous record — likely reflects extended internal reasoning on novel problem types. Benchmark scores derived with unconstrained thinking budgets may not reflect costs under production budget_tokens limits.

Open-weight models (hosted pricing)

Self-hosted = free. These are representative hosted prices from providers like Groq, Together AI, Fireworks, and Alibaba Cloud.

Model	Hosted Input/M	Hosted Output/M	Context	License
Tencent Hy3 (295B)	~$0.06	~$0.06	—	Open
Llama 4 Scout (109B/17B active)	~$0.10	~$0.40	10M	Meta
Qwen3.6-27B	~$0.12	~$0.50	128K	Apache 2.0
Llama 4 Maverick	~$0.15	~$0.60	1M	Meta
Qwen3.6-35B-A3B	~$0.15	~$0.60	128K	Apache 2.0
Gemma 3 27B	~$0.20	~$0.20	128K	Google
Mistral Small 4	$0.15	$0.60	256K	Apache 2.0
Poolside Laguna S 2.1	~$0.30	~$1.00	—	Open-weight
DeepSeek R1 (hosted)	$0.55	$2.19	128K	MIT
GLM-5.2	~$0.50	~$2.00	—	Open-source
Command A+ (218B MoE, 25B active)	~$0.50	~$1.50	256K	Apache 2.0
Devstral 2 (123B)	$0.40	$2.00	256K	Apache 2.0
Kimi K3 (2.8T)	~$1.00	~$3.00	1M	Open
Thinky Inkling (975B)	~$0.80	~$2.50	—	Apache 2.0
Mistral Large 3 (675B MoE, hosted)	~$0.50	~$1.50	256K	Apache 2.0

Kimi K3 (Moonshot AI) is the largest open model released to date at 2.8T parameters, with a 1M context window and Opus-class performance at approximately Sonnet-level pricing. Demand was high enough that Moonshot paused new subscriptions within 48 hours of launch.

Thinky Inkling (975B, multimodal, Apache 2.0) is reported to lead US labs on certain benchmarks. Hosted pricing is from third-party inference providers; self-hosted requires substantial GPU capacity.

Tencent Hy3 (295B, open-weights) at roughly $0.06/M tokens is among the cheapest capable options for workloads it handles well. It is available on several third-party hosting platforms.

GLM-5.2 is a notable open-source Chinese model in production use — Databricks reported it matched Claude Opus 4.8 on their internal codebase at lower cost.

Poolside Laguna S 2.1 is a compact open-weight coding model with reported strength in self-correction and persistence on long agentic tasks. Hosted pricing is approximate from available inference providers.

DeepSeek V4-Flash 0731 is an updated variant of V4-Flash that reached within one point of GPT-5.6 Luna on the Artificial Analysis Intelligence Index at approximately 60% lower cost per task. It replaces the earlier V4-Flash as the primary DeepSeek efficiency reference.

Qwen3.8-Max is API-only on Alibaba Cloud Model Studio today, with a weights release scheduled but not yet shipped. Until it lands, the open-weight Qwen3.6 family (including Qwen3.6-35B-A3B with 3B active parameters) remains the primary self-hosting target within that family for cost-constrained inference.

Embeddings and other modalities

Model	Price	Unit	Notes
OpenAI text-embedding-3-small	$0.02/M tokens	tokens
OpenAI text-embedding-3-large	$0.13/M tokens	tokens
Gemini Embedding 2	Free (AI Studio)	tokens	Multimodal (text, image, video, audio)
Cohere Embed v3	$0.10/M tokens	tokens	Multilingual
Whisper (OpenAI)	$0.006/min	audio minutes
GPT Transcribe / GPT Live Transcribe	TBD	audio minutes	Now available via API; independent benchmarks show higher error rates than ElevenLabs, Google, and Mistral
Whisper (Groq, free tier)	$0.00	audio minutes	8 hrs/day limit
Gemini 3.1 Flash TTS	Contact Google	characters	Dedicated TTS model
Gemini Omni	Contact Google	—	Multimodal any-input; launched Google I/O 2026
DeepMind Lyria 3.5	Contact Google	—	Music generation; integrated into Google Flow Music
OpenAI TTS	$15.00/M chars	characters
OpenAI TTS HD	$30.00/M chars	characters
GPT Image 2	$0.02–$0.17	per image	Varies by quality/size

The hidden cost multipliers

The pricing table above is the starting point, not the answer. Several factors multiply your actual spend beyond the per-token rate.

1. Output tokens cost 2–6x more than input tokens

Every major provider charges more for output than input. The ratio varies:

Provider/Model	Typical Output:Input Ratio
DeepSeek V4-Flash 0731	2:1
Grok 4.5	3:1
Grok 4.3	2:1
Qwen3.7-Max	3:1
GPT-5.4	4:1
Claude Sonnet 5	5:1
Claude Opus 5	5:1
GPT-5.5	6:1

Prompt engineering that reduces output length has an outsized cost impact. A system prompt that gets the model to return 200 tokens instead of 500 tokens saves more than a system prompt that reduces input by the same 300 tokens.

2. Context length pricing tiers

Google charges different rates based on how much of the context window you use. Gemini 3.1 Pro costs $2/$12 per million tokens under 200K, but $4/$18 above — a 50% price increase for long-context workloads. Gemini 3.6 Flash’s flat pricing regardless of context depth is a practical advantage for workloads that push toward the 1M token window. Other providers charge flat rates regardless of context usage.

3. Thinking token overhead

Extended thinking is not free. A request to Claude Opus 5 that triggers 20,000 thinking tokens at $25/M output adds $0.50 to that single request before the actual response tokens are counted. For complex reasoning tasks, thinking tokens can exceed response tokens by 10–50x. Opus 5’s ARC-AGI-3 performance almost certainly involves substantial thinking-token use — benchmark scores at unconstrained thinking budgets will not reflect production costs under budget_tokens limits.

4. Cache write costs

Anthropic charges a 25% premium on tokens written to cache versus standard input. Claude Opus 5: $6.25/M for cache writes versus $5.00/M for standard input. The write premium pays for itself after the second cache read ($0.50/M), so any cached prefix used 3+ times is net positive. Single-use prompts should skip caching. Claude Sonnet 5 follows the same pattern at $3.75/M cache write versus $3.00/M standard input.

5. Claude Sonnet 5 tokenizer inflation — and the end of introductory pricing

Sonnet 5’s tokenizer emits approximately 30% more tokens for the same text than the tokenizer used in Sonnet 4.6. Once introductory pricing ends on August 31, the effective per-character cost of Sonnet 5 at $3/$15 is meaningfully higher than Sonnet 4.6 at the same list price. A rough estimate: the tokenizer inflation plus list-price parity implies Sonnet 5 costs approximately 30% more per unit of text than Sonnet 4.6 at equivalent pricing. Whether that premium is justified by Sonnet 5’s quality improvements depends on the workload — but any cost model that assumed introductory-period parity or tokenizer-ignoring comparisons should be re-benchmarked on actual workload data before the switchover.

6. Image and multimodal token costs

Images are converted to tokens before processing. A single high-resolution image can consume 1,000–5,000+ tokens depending on the model and resolution settings. For vision-heavy workloads — document OCR, screenshot analysis, diagram interpretation — image token costs can dominate the bill. Resize images to the minimum resolution needed before sending them. Gemini 3.6 Flash’s agentic and multimodal positioning, combined with Gemini 3.5 Flash’s computer-use capability (screenshots, clicking, UI navigation), creates a pattern where a single agentic session can accumulate dozens of screenshot tokens across a task sequence. Budgeting per-session rather than per-request is more accurate for those workloads.

7. Autonomous operation and agentic loops

An agent that reads its own output history accumulates input tokens roughly quadratically over a long session. Qwen3.7-Max’s 35-hour autonomous operation demonstration illustrates the ceiling — a session of that length, even at $2.50/M input, could accumulate millions of context tokens across iterative steps. Claude Code Auto Mode (now the default for Claude Code) introduces similar dynamics: its safety classifier processes each command, and multi-step coding tasks generate context that grows with each tool call. Claude Code instances running in parallel on macOS and Linux can now message each other and share context across terminals, which creates multi-agent context accumulation patterns that are difficult to budget without session-level token caps. Budget token limits at the task level, not just the request level.

8. GPT-5.6 pricing: the cut comes from tiering, not from the flagship

OpenAI’s “up to 80%” cost reduction claim holds only if you move down the family. Sol lists at $5/$30 per million tokens — identical to GPT-5.5, not cheaper. The savings live in the lower tiers: Terra at $2/$12 is roughly 60% below GPT-5.5, and Luna at $0.20/$1.20 is about 96% below it. Read against the flagship you were already using, the headline number describes a tiering strategy rather than a price cut on equivalent capability. Budget from the specific variant you plan to call, not from the family-level claim.

9. Access-gated models

GPT-5.4-Cyber is available only to vetted teams working on defensive security. Claude Mythos 5 is restricted to vetted US organizations via Project Glasswing. GPT-5.6 Sol has had restricted US access beyond the initial rollout. None have publicly documented per-token costs. Workload planning that assumes access to any of these models requires factoring in the vetting or partnership process separately from cost modeling.

Prompt caching: how it works, what it saves

Prompt caching stores the processed representation of input tokens on the provider side so subsequent requests with the same prefix skip re-processing.

Provider comparison

Provider	Cache Read Discount	Cache Write Premium	Min Cacheable	TTL	Rate Limit Benefit
Anthropic	90% off input	25% over input	1,024 tokens	5 min (auto-extends)	Cached reads don’t count toward ITPM
OpenAI	90% off input	None (automatic)	Varies	Automatic	No rate limit benefit
Google	75% off input	None (usage-based)	4,096 tokens	Configurable	No rate limit benefit
DeepSeek	80–90% off input	None	Automatic	Varies	No rate limit benefit

Anthropic’s caching has the most generous rate limit interaction: cached read tokens are excluded from ITPM limits entirely. With 80% cache hits, a 2M ITPM limit effectively handles 10M total input tokens per minute. This is a throughput multiplier, not just a cost reduction.

Claude Opus 5 cache pricing ($0.50/M reads, $6.25/M writes) follows the same structural discount ratios — 90% off input for reads, 25% premium for writes — scaled to the higher list price. The tokenizer inflation from Sonnet 5 applies to Opus 5 as well if workloads involve substantial natural language prompting; re-benchmark cached prefix sizes on actual prompts when moving between Anthropic models.

When caching pays off

The break-even math for Anthropic: cache write costs 25% more than standard input. Cache reads cost 90% less. After 2 cache reads, even at a standard 25% write premium, the cost is net positive. For a system prompt reused across thousands of requests, the savings approach 90% on that portion of every request.

Practical caching patterns

RAG with stable context: Cache the system prompt and retrieved documents as a prefix. Each follow-up query appends to the cached prefix. Works well when the same document set is queried multiple times within the TTL window.

Multi-turn conversations: Cache the conversation history prefix. Each new turn extends the cache. Anthropic’s 5-minute auto-extending TTL means active conversations stay cached indefinitely.

Batch evaluation: Cache the evaluation rubric and few-shot examples. Run hundreds of evaluations against the cached prefix. The rubric tokens (often 2,000–5,000) are paid once at write price, then read at 90% discount for every subsequent evaluation.

Agentic task context: For long-running autonomous tasks, cache the task description, tool definitions, and accumulated working memory up to the stable prefix boundary. Claude Code Auto Mode sessions, Gemini 3.6 Flash agentic tasks via Managed Agents, and Qwen3.8-Max long-horizon runs all benefit from this pattern. For Gemini 3.6 Flash computer-use sessions, caching the stable task description and tool schema before the screenshot stream begins can meaningfully reduce costs on long UI-navigation tasks. Claude Code instances running parallel on multiple terminals compound the benefit — a cached shared context is read multiple times simultaneously.

Batch APIs: the easiest 50% discount

OpenAI, Anthropic, and Google all offer batch processing with lower pricing and higher throughput limits.

Provider	Batch Discount	Turnaround	Queue Limits
OpenAI	50% off standard pricing	Up to 24 hours	3x–100x higher than real-time TPM
Anthropic	50% off standard pricing	Up to 24 hours	Higher than real-time
Google	50% off standard pricing	Up to 24 hours	Available for Gemini 3.1+ models

OpenAI’s batch queue limits are notably generous: at Tier 5, the batch queue reaches 15 billion tokens. The 50% discount applies to whichever pricing tier is in effect at submission time.

For Claude Sonnet 5 at standard pricing, batch pricing is $1.50/$7.50 per million input/output — still the most accessible entry point to Sonnet 5-class quality. For Claude Opus 5 at $5.00/$25.00 standard, batch pricing of $2.50/$12.50 positions it closer to Sonnet or Opus 4.8 territory on a per-token basis, which changes the routing calculus for non-real-time complex workloads.

Workloads that fit batch processing: content generation pipelines, bulk classification, evaluation suites, data extraction from document archives, nightly report generation, synthetic data creation. Any workload where the output is consumed minutes or hours after generation rather than in real time.

Model routing: the architecture that pays for itself

The most impactful cost optimization is not using one model for everything. A routing layer that sends requests to different models based on complexity can cut spend 60–85%.

Cost comparison: routed vs. single-model

Assume 100,000 requests/day, average 500 output tokens per request.

Single model (Claude Sonnet 5, standard pricing):

100K requests × 500 tokens = 50M output tokens/day
50M × $15/M = $750/day

Note: Sonnet 5’s tokenizer produces ~30% more tokens for equivalent output compared to Sonnet 4.6. Actual token counts per request should be measured rather than assumed from prior workload benchmarks.

Three-tier routing (70/25/5 split):

70K requests × 500 tokens × $5/M (Haiku 4.5) = $175.00
25K requests × 500 tokens × $15/M (Sonnet 5, standard) = $187.50
5K requests × 500 tokens × $25/M (Opus 5) = $62.50
Total: $425.00/day (43% savings over Sonnet 5 alone)

The classifier itself costs almost nothing. Running GPT-4.1 Nano ($0.10/M input) or Haiku 4.5 ($1.00/M input) to classify intent on a 100-token input across 100K daily requests works out to 10M input tokens per day: roughly $1–10/day.

For coding-specific routing, Fable 5 remains the clearest complex-tier target at 95% SWE-bench — Opus 5 matches its coding performance while costing 50% less per task on knowledge tasks, making it a reasonable alternative depending on the mix of coding versus reasoning in the complex tier.

For factual accuracy-constrained workloads, Grok 4.5 at $6.00/M output deserves evaluation as the medium tier. Its independent evaluation at Opus-class quality sits between Haiku 4.5 and Sonnet 5 on output cost, making it worth benchmarking as a medium-complexity routing target.

Open-weight routing for cost floors

For teams with GPU infrastructure, routing the simple tier to a self-hosted model (Llama 4 Scout, Qwen3.6-35B-A3B, Tencent Hy3, Command A+) eliminates the per-token cost entirely on the majority of requests. Kimi K3’s 2.8T parameter scale makes it impractical for typical self-hosting, but hosted pricing at approximately Sonnet-level cost makes it a candidate for the medium tier in routing pipelines. Command A+‘s Apache 2.0 license and 2×H100 inference footprint (or single B200) remain practical for the medium tier at near-zero marginal cost above infrastructure. Devstral 2’s 72.2% SWE-bench score holds as the dedicated open-weight coding tier — the gap to Fable 5 on hard coding problems is now substantial, but Devstral 2 remains competitive with mid-tier proprietary models on most code tasks at self-hosted inference costs.

Token efficiency: not all tokens are equal

The same task produces different token counts on different models. This is the most underappreciated cost variable.

GPT-5.6’s efficiency improvement

OpenAI’s recursive self-improvement and distillation process, which they credit for the 80% price reduction in GPT-5.6 relative to GPT-5.5, implies substantial efficiency gains. GPT-5.5 was already reported as ~40% more token-efficient than GPT-5.4 on coding tasks; whether GPT-5.6 compounds that further or achieves the same output quality with fewer generation steps is not yet documented in third-party evaluations. The DeepSeek V4-Flash 0731 result — within one point of GPT-5.6 Luna on the Artificial Analysis Intelligence Index at ~60% lower cost per task — suggests that per-token price cuts alone do not guarantee task-level cost leadership.

Claude Opus 5 task-level economics

Anthropic’s claim that Opus 5 costs roughly 50% less per task than Fable 5 on knowledge-intensive workloads is backed directly by list pricing: at $5/$25 per million tokens versus Fable 5’s $10/$50, Opus 5 is half the price per token at near-Fable-5 intelligence. That makes the per-task saving the straightforward case rather than one that depends on token efficiency — but any additional saving from Opus 5 reaching the correct answer in fewer reasoning steps or fewer retry loops still needs to be measured on representative workloads before it is assumed in a budget.

Claude Sonnet 5 tokenizer effects at standard pricing

Once introductory pricing ends, the tokenizer inflation compounds the list-price increase. Moving from Sonnet 4.6 at $3/$15 to Sonnet 5 at $3/$15 with ~30% more tokens per equivalent text results in approximately 30% higher per-task cost at equivalent quality — before accounting for any quality improvements that might allow shorter outputs. Whether Sonnet 5 is cheaper per task than Sonnet 4.6 depends entirely on the ratio of quality improvement (allowing fewer tokens or fewer retries) to tokenizer inflation on a given workload.

Grok 4.5’s per-task cost position

At $2.00/$6.00 per million tokens with a 3:1 ratio, Grok 4.5 sits below the frontier flagships on the cost curve for a model with Opus-class performance claims. For workloads where factual accuracy or instruction following is the binding quality constraint, it is worth benchmarking on a per-task basis rather than relying on per-token comparisons alone. The pricing figures are provisional — confirm before production deployment.

Tokenizer differences matter

Different providers use different tokenizers. The same English sentence might be 15 tokens on OpenAI’s tiktoken (cl100k_base), 18 tokens on Anthropic’s tokenizer for Sonnet 4.6, and approximately 23 tokens on Sonnet 5’s tokenizer. For pricing comparisons across providers, convert to a common unit (characters or words) rather than comparing raw per-token prices.

A rough heuristic: 1 token ≈ 4 characters ≈ 0.75 words in English. This varies by language (CJK characters often consume 1–2 tokens each) and by content type (code is tokenized differently than prose). The Sonnet 5 tokenizer shifts this ratio to approximately 1 token ≈ 3 characters in English.

Prompt engineering for token efficiency

Small changes to prompts can produce large token savings at scale:

“Respond in JSON” vs. natural language: structured responses are 30–60% shorter.
“Be concise” in the system prompt: an explicit brevity instruction reduces output tokens by 20–40% on average.
Few-shot examples: Adding 2–3 examples of desired output length calibrates verbosity. This costs a few hundred extra input tokens but saves thousands of output tokens across a batch.
Max tokens parameter: Set a hard cap. Prevents runaway generation on edge cases where the model would otherwise produce 10x the expected output.

Structured outputs: fewer tokens, same information

JSON mode and structured outputs (GA on OpenAI, Anthropic, and Google) constrain the model to return data in a predefined schema. This is both a reliability and a cost optimization.

A sentiment classification task:

Unstructured response (typical): “The sentiment of this review is positive. The customer expresses satisfaction with the product quality and delivery speed, though they note minor concerns about packaging.” (~35 tokens)

Structured response: {"sentiment": "positive", "confidence": 0.92} (~12 tokens)

That is a 65% reduction in output tokens per request. At $15/M output (Sonnet 5, standard pricing), across 1M daily classifications, the savings are: (35−12) × 1M / 1M × $15 = $345/day.

For any pipeline where the downstream consumer is code rather than a human reading prose, structured outputs should be the default.

Self-hosting vs. API: when the math flips

Open-weight models (Llama 4, Qwen3.6, Mistral, DeepSeek, Command A+, Devstral 2, Kimi K3, Thinky Inkling, Tencent Hy3) are free to run. The cost is infrastructure.

Break-even estimation

A single H100 (80GB) rents for roughly $2–3/hour on cloud providers. It can serve a 70B model (quantized to 4-bit) at approximately 30–50 tokens/second.

Monthly H100 cost: ~$2,000 Monthly tokens at 40 tok/s sustained: ~100B tokens

Equivalent API cost for 100B output tokens:

Claude Haiku 4.5: $500,000
Claude Sonnet 5 (standard): $1,500,000
GPT-4.1 Nano: $40,000
DeepSeek V4-Flash 0731: $28,000
Grok 4.5: $600,000

Self-hosting a 70B open model at $2,000/month breaks even against DeepSeek V4-Flash 0731 at roughly 7B output tokens/month. Against Haiku 4.5, it breaks even at under 500M tokens/month.

Kimi K3 (2.8T parameters) is too large for most self-hosting setups — it would require a substantial multi-GPU cluster. Hosted pricing at approximately $3/M output puts it near Gemini 3.6 Flash territory economically, positioning it as an API option rather than a self-hosting target for most teams.

Thinky Inkling (975B, Apache 2.0) is more feasible to self-host than Kimi K3 but still requires significant multi-GPU infrastructure. For teams already operating H100 clusters at scale, its Apache 2.0 license and reported benchmark leadership over US labs make it worth evaluating.

Command A+ changes the calculus meaningfully. At 218B total parameters but only 25B active, it runs on 2×H100 (~$4,000/month) while delivering quality closer to mid-tier proprietary models than typical 70B open-weights. The B200 option reduces infrastructure cost further for teams with access to newer hardware.

Devstral 2 occupies a narrow but useful niche: a 72B coding-focused model with Apache 2.0 licensing and 256K context that runs on a single H100. For coding-heavy workloads, its 72.2% SWE-bench performance at open-weight inference costs remains difficult to match with API alternatives at comparable volume — though teams needing Fable 5-class performance (95% SWE-bench) on hard coding problems have no open-weight substitute yet.

When APIs still win

Low volume: Under ~5B tokens/month, API costs are lower than maintaining infrastructure.
Burst traffic: APIs absorb spikes. Self-hosted GPUs sit idle during troughs and cannot handle peaks without over-provisioning.
Frontier quality: No open model approaches Claude Fable 5 or Opus 5 on complex coding, or matches Opus 5’s ARC-AGI-3 score on novel reasoning. Tasks requiring the best available performance leave no self-hosted alternative.
Compliance and support: Enterprise API tiers include SLAs, BAAs (HIPAA), data residency controls, and zero-retention options that would require substantial additional investment to replicate on self-hosted infrastructure.

When self-hosting wins

High sustained volume: Above 10B+ tokens/month, owned or reserved GPU capacity typically beats API pricing — particularly now that GPU hardware costs are declining.
Data sovereignty: Air-gapped or on-premise requirements.
Fine-tuning: Self-hosted models can be fine-tuned on proprietary data with full control. API-based fine-tuning is available from some providers but with less flexibility.
Latency control: No network round-trip. Co-located inference can achieve sub-10ms TTFT.

Cost modeling a real workload

Pricing tables are abstractions. A concrete example of modeling costs for a production RAG pipeline follows.

Scenario: customer support agent

50,000 queries/day
Each query: 200 tokens input (user question), 2,000 tokens retrieved context, 300 tokens system prompt, 400 tokens output
System prompt + retrieval context is 80% cacheable (same FAQ docs across queries)

Important: These token counts are measured on a Sonnet 4.6-tokenized workload. If running on Sonnet 5, the ~30% tokenizer inflation means 2,500 input tokens likely becomes ~3,250 input tokens for the same text. The output column is similarly affected. Re-measure on actual workload data before committing to a cost model.

Option A: Claude Sonnet 5 (standard pricing), no optimization

Component	Tokens/request	Daily tokens	Cost/M	Daily cost
Input (system + context + query)	2,500 (est. 3,250 actual on Sonnet 5)	162.5M	$3.00	$487.50
Output	400 (est. 520 actual on Sonnet 5)	26M	$15.00	$390.00
Total				$877.50/day

Monthly: ~$26,325. The delta versus the July edition’s Option A ($450/day on introductory pricing) illustrates the combined effect of the introductory period expiring and tokenizer inflation on an unmigrated cost model.

Option B: Claude Sonnet 5, with prompt caching

Component	Tokens/request	Daily tokens	Cost/M	Daily cost
Cache write (system + context, first request/5min)	2,300 (est. ~2,990 actual)	~0.9M	$2.50	$2.25
Cache read (system + context, subsequent)	2,300 (est. ~2,990 actual)	~143.5M	$0.30	$43.05
Uncached input (user query)	200 (est. ~260 actual)	13M	$3.00	$39.00
Output	400 (est. ~520 actual)	26M	$15.00	$390.00
Total				$474.30/day

Monthly: ~$14,229. 46% savings from caching alone — the ratio holds even as absolute numbers rise with tokenizer inflation.

Option C: Routed, with caching

Route 70% of queries (simple FAQ-type) to Haiku 4.5, 30% (complex) to Sonnet 5. Both with caching.

Component	Daily cost
Haiku tier (35K queries, cached)	~$52.50
Sonnet tier (15K queries, cached)	~$151.50
Classifier (50K queries, Haiku)	~$0.50
Total	~$204.50/day

Monthly: ~$6,135. 77% savings vs. Option A.

Option D: Add batch processing for non-real-time queries

If 20% of queries (overnight analytics, report generation) can tolerate 24-hour turnaround:

Component	Daily cost
Real-time routed (40K queries)	~$163.60
Batch (10K queries, 50% discount)	~$20.45
Total	~$184.05/day

Monthly: ~$5,521. 79% savings vs. Option A.

The progression from ~$26,325/month to ~$5,521/month uses no different models, no degradation in output quality, and no self-hosting. It is entirely a function of caching, routing, and batching. The absolute numbers are higher than in the July edition due to the Sonnet 5 introductory period ending and tokenizer inflation, but the relative savings from each optimization layer remain roughly constant.

Quick reference: cost per 1M output tokens

Sorted cheapest to most expensive. Input costs excluded.

Model	Output $/M	Type
Tencent Hy3 (295B)	~$0.06	Open-weight
DeepSeek V4-Flash 0731	$0.28	Proprietary
Llama 4 Scout (hosted)	~$0.40	Open-weight
DeepSeek V4-Pro	$0.87	Proprietary
GPT-5.5 Instant	$1.00	Proprietary
GPT-5.6 Luna	~$1.20	Proprietary
GPT-4.1 Nano	$0.40	Proprietary
Gemini 3.1 Flash-Lite	$1.50	Proprietary
DeepSeek R1	$2.19	Open-weight
Grok 4.3	$2.50	Proprietary
Gemini 3.6 Flash	$3.75	Proprietary
Muse Spark 1.1	$4.25	Proprietary
Claude Haiku 4.5	$5.00	Proprietary
Grok 4.5	$6.00	Proprietary
Qwen3.7-Max	$7.50	Proprietary
Gemini 3.5 Flash	$9.00	Proprietary
GPT-5.4 Thinking	$12.00	Proprietary
GPT-5.6 Terra	$12.00	Proprietary
Gemini 3.1 Pro	$12–18	Proprietary
GPT-5.4	$15.00	Proprietary
Claude Sonnet 4.6	$15.00	Proprietary
Claude Sonnet 5	$15.00	Proprietary
Claude Opus 4.8	$25.00	Proprietary
Claude Opus 5	$25.00	Proprietary
GPT-5.5	$30.00	Proprietary
GPT-5.6 Sol	$30.00	Proprietary
GPT-5.5 Thinking	$36.00	Proprietary
Claude Fable 5	$50.00	Proprietary