API Rate Limits Compared: Every Major LLM Provider (July 2026)

Eighth edition. The seventh (June 13, 2026) covered 17 providers. This update reflects the four-week window since June 13. Claude Sonnet 5 is now Anthropic’s balanced tier, replacing Sonnet 4.6 in entry-paid and scaled tables. Claude Fable 5 has been restored globally as of July 1 after a brief export-control suspension. xAI’s current flagship is Grok 4.3, replacing Grok 4.20. Gemini 3.5 Flash is now GA. The DeepSeek V3.2 alias deprecation is two weeks out (July 24).

What changed since June 13
Free Tier — RPM & RPD
Free Tier — TPM & TPD
Entry Paid Tier — RPM & RPD
Entry Paid Tier — TPM
Scaled Tier — RPM & TPM
Cerebras & SambaNova
Audio Models — ASH & ASD
Cloud Aggregators — Azure AI & AWS Bedrock
More Providers — Perplexity, Alibaba, Moonshot
Provider Notes
Tips for managing rate limits
Further Reading

Last updated: July 10, 2026.

What changed since June 13

Claude Sonnet 5 is Anthropic’s new balanced tier. Released June 30, 2026, Sonnet 5 closes much of the gap to Opus 4.8 at the balanced tier price point. Its new tokenizer emits approximately 30% more tokens for the same text — relevant for TPM planning, since your token counts will be higher than with Sonnet 4.6 for identical prompts. Introductory pricing ($2/$10 per M input/output) runs through August 31, 2026, then shifts to standard $3/$15. Rate limit structure appears to follow the same tier profile as Sonnet 4.6; Sonnet 5 and 4.6 likely share a pool (not yet formally confirmed in Anthropic’s docs — check your dashboard). Sonnet 4.6 remains available.

Claude Fable 5 restored globally July 1. Fable 5 was released June 9, suspended June 12 under a US export-control directive citing a jailbreak vulnerability, and restored July 1 after the controls were lifted June 30. On redeployment, Anthropic added a classifier that blocks the reported exploit technique and reroutes flagged requests to Opus 4.8. Fable 5 is now available on the Claude Platform, Claude.ai, Claude Code, and through AWS/Google Cloud/Microsoft Foundry. Rate limit figures for Fable 5 are not yet published — it is not represented in tables below. Claude Mythos 5 remains restricted to vetted US organizations via Project Glasswing and is not generally available.

xAI flagship is now Grok 4.3. Released April 30, 2026 (post-dates the June 13 edition — this was an oversight in the previous entry, which still listed Grok 4.20). Grok 4.3 leads on non-hallucination rate, agentic tool calling, and instruction following. xAI continues to gate numerical RPM/TPM behind the console. Tables updated to reference Grok 4.3.

Gemini 3.5 Flash is now GA. Released at Google I/O 2026 (May 19), it was listed as having no published rate limit figures in the June 13 edition. It is now GA and is Google’s current flagship, leading on agentic and coding benchmarks with approximately 4x faster output than other frontier models. Rate limit figures remain absent from Google’s public documentation — check AI Studio (aistudio.google.com/rate-limit) for your project’s actual limits. Gemini 3.5 Pro remains in limited preview.

DeepSeek V3.2 alias deprecation is two weeks out. July 24, 2026. V4-Flash and V4-Pro are the current endpoints. Any pipeline still referencing a V3.2 alias needs to be updated now.

OpenAI GPT-5.6 Sol previewed June 27. Limited early access only — tiered variants Sol/Terra/Luna rolling out to trusted partners. Not generally available and not represented in rate limit tables. US access is further restricted. GPT-5.5 remains the current flagship for API access.

Still pending formal documentation:

Alibaba: Qwen3.7-Max (May 20) rate limit tables still not published. Qwen3 Max limits continue as a provisional floor.
Cohere: Command A+ (May 20) rate limit figures still not published. Command R+ figures remain as a reference baseline.
Google: Gemini 3.5 Flash and Gemini 3.5 Pro rate limit tables not yet published in public docs. Gemini Omni figures also absent.
Anthropic: Claude Fable 5 rate limit figures not yet published.

Deprecation calendar:

DeepSeek V3.2 aliases deprecate July 24, 2026 — two weeks out. V4-Flash and V4-Pro are the current endpoints. Update hardcoded model references now.
OpenAI Assistants API deprecates August 2026; replaced by the Responses API.

What are rate limits?

Rate limits cap how much you can use an API within a given time window. Providers enforce them to manage capacity and ensure fair access. Exceeding a limit returns a 429 Too Many Requests error.

Most providers use the token bucket algorithm: capacity refills continuously up to your maximum, rather than resetting at fixed intervals. A 60 RPM limit means roughly 1 request per second with steady refill, not 60 requests then a hard stop.

The metrics

RPM (Requests per minute): API calls per minute, regardless of size. A 10-token request and a 100K-token request both count as one.
RPD (Requests per day): Daily cap on total API calls. Some providers use this instead of (or alongside) RPM, especially on free tiers.
TPM (Tokens per minute): Total tokens (input + output) processed per minute. Usually the binding constraint in production.
TPD (Tokens per day): Daily token cap. More common on free tiers.
ITPM/OTPM (Input/Output tokens per minute): Anthropic separates input and output token limits, giving finer control. Cached input tokens don’t count toward ITPM on current Claude 4.x and 5.x models.
ASH/ASD (Audio seconds per hour/day): For speech models like Whisper.

RPM limits set your concurrency ceiling. TPM limits set your throughput. For batch workloads, RPD and TPD matter more. For real-time apps, RPM and TPM hit first.

Free Tier — Requests per Minute (RPM) & Requests per Day (RPD)

Provider / Model	RPM	RPD
Google — 3.1 Flash-Lite	15	1,000
Groq — Llama 4 Scout	30	1,000
Cerebras — Llama 4 Scout	30	—
SambaNova — open models	10–30	—
Fireworks AI — OS models	10	—
Cohere — Command R+*	20	~33†

*Cohere’s free tier figures reference Command R+. Command A+ free tier limits not yet published. †Cohere trial keys get 1,000 calls/month across all endpoints, roughly ~33/day.

Note: Older Gemini tiers have conservative limits and are not listed here. Check AI Studio for your project’s actual caps. Gemini 3.5 Flash free tier limits are not yet published in public documentation. OpenAI, Anthropic, xAI, DeepSeek, and Mistral have no free API tier for current flagship models. OpenAI’s free tier does not support any GPT-5.x model via the API (GPT-5.5 Instant is a ChatGPT product, not an API tier). Anthropic’s lowest entry is $5 (Tier 1). xAI and Mistral gate numerical limits behind their respective consoles.

Migration note: Gemini 2.0 Flash and Flash-Lite are shut down as of June 1, 2026. Those endpoints no longer accept requests.

Free Tier — Tokens per Minute (TPM) & Tokens per Day (TPD)

Provider / Model	TPM	TPD
Google — 3.1 Flash-Lite	250,000	—
Groq — Llama 4 Scout	30,000	500,000
Groq — other open models	6,000–30,000	500,000
Cerebras — Llama 4 Scout	60,000	1,000,000
SambaNova (free tier)	—	—

Standout: Google leads with 250K TPM on the free tier, but the RPM/RPD caps (10–15 RPM) mean it’s best for fewer, larger requests. Gemini 3.5 Flash free tier TPM figures are not yet published. Cerebras offers 60K TPM with 1M TPD and approximately 2,100 tokens/second inference speed on Llama 4 Scout, making it the fastest free option by throughput. SambaNova publishes RPM but not TPM/TPD specifics.

Entry Paid Tier — Requests per Minute (RPM) & Requests per Day (RPD)

The tier most developers start with. OpenAI Tier 1 ($5), Anthropic Tier 1 ($5), Google Tier 1 (pay-as-you-go), and others at their lowest paid level.

Provider / Model	RPM	RPD
OpenAI — GPT-5.5	500	—
OpenAI — GPT-5.4 Nano	500	—
Anthropic — Sonnet 5	50	—
Anthropic — Haiku 4.5	50	—
Google — 3.1 Pro	150	1,000
Google — 3.1 Flash-Lite	300	1,500
xAI — Grok 4.3	Console†	—
DeepSeek — V4 Flash	Dynamic‡	—
Fireworks AI — OS models	≤6,000	—
Cohere — Command A+*	500	—

*Command A+ entry paid tier limits not yet published. 500 RPM is the Command R+ figure, carried forward as a reference. †xAI publishes tier thresholds ($0/$50/$250/$1K/$5K) but numerical RPM/TPM are only visible in the xAI Console after login. ‡DeepSeek uses fully dynamic concurrency limits based on server load. No fixed RPM/TPM published.

Gemini 3.5 Flash note: Gemini 3.5 Flash is now GA and is Google’s current flagship. Published rate limit figures are not available in public documentation — check AI Studio for your project’s actual limits. Gemini 3.5 Pro remains in limited preview. Gemini 3.1 Pro and 3.1 Flash-Lite remain available as the previous generation, with the last confirmed published figures shown above.

Mistral note: Mistral moved to console-only limits. Actual numbers require login at admin.mistral.ai/plateforme/limits. Mistral is excluded from RPM columns where no public figure exists.

Standout: Fireworks can spike to 6,000 RPM but it’s a dynamic ceiling, not guaranteed (soft limit starts at approximately 1 RPS and doubles hourly). Anthropic’s 50 RPM at Tier 1 is the lowest here, but jumps 20x to 1,000 RPM at Tier 2 ($40). Note that Sonnet 5’s tokenizer emits approximately 30% more tokens per request than Sonnet 4.6 for identical prompts — TPM headroom shrinks faster than it did on 4.6 at the same RPM.

Entry Paid Tier — Tokens per Minute (TPM)

Provider / Model	TPM
OpenAI — GPT-5.5	500K
OpenAI — GPT-5.4 Nano	200K
Anthropic — Sonnet 5	30K in / 8K out
Anthropic — Haiku 4.5	50K in / 10K out
Google — 3.1 Pro	1M
Google — 3.1 Flash-Lite	2M
DeepSeek — V4 Flash	Dynamic

Standout: Google’s 3.1 Flash-Lite leads the confirmed figures at 2M TPM. GPT-5.5 gets 500K TPM at Tier 1. Anthropic’s Sonnet 5 shows 30K ITPM — identical to Sonnet 4.6’s figure, but the tokenizer change means identical prompts now consume more tokens against that limit. Cached input tokens still don’t count toward ITPM on Claude 4.x and 5.x models — with an 80% cache hit rate, effective throughput remains significantly higher than the raw ITPM suggests.

Scaled Tier — RPM & TPM at Higher Spend

For teams past entry-level. OpenAI Tier 3 ($100+), Anthropic Tier 3 ($200+), Google Tier 2 ($250+).

Provider (Tier) / Model	RPM	TPM
OpenAI (Tier 3) — GPT-5.5	5,000	2M
OpenAI (Tier 3) — GPT-5.4 Nano	5,000	4M
Anthropic (Tier 3) — Opus 4.8	2,000	800K in / 160K out
Anthropic (Tier 3) — Sonnet 5	2,000	800K in / 160K out
Anthropic (Tier 3) — Haiku 4.5	2,000	1M in / 200K out
Google (Tier 2) — 3.1 Pro	1,000	2M
Google (Tier 2) — 3.1 Flash-Lite	2,000	4M

Sonnet 5 Tier 3 TPM figures are carried forward from Sonnet 4.6. Confirm current limits in the Anthropic console — pool assignment with Sonnet 4.6 is not yet formally documented.

At the highest standard tiers (OpenAI Tier 5 at $1,000+, Anthropic Tier 4 at $400+):

Provider (Tier) / Model	RPM	TPM
OpenAI (Tier 5) — GPT-5.5	15,000	40M
OpenAI (Tier 5) — GPT-5.4 Nano	30,000	180M
Anthropic (Tier 4) — Opus 4.8	4,000	2M in / 400K out
Anthropic (Tier 4) — Sonnet 5	4,000	2M in / 400K out
Anthropic (Tier 4) — Haiku 4.5	4,000	4M in / 800K out

Sonnet 5 Tier 4 TPM figures carried forward from Sonnet 4.6. Check the Anthropic console for confirmed limits.

Standout: OpenAI’s Tier 5 numbers remain the highest of any provider with published figures. GPT-5.4 Nano at 180M TPM and 30K RPM is built for high-volume classification and routing. Anthropic’s Tier 4 caps at 4K RPM, but cached tokens don’t count toward ITPM — effective throughput can be substantially higher with good cache hit rates. The Sonnet 5 tokenizer change means any existing throughput calculations based on Sonnet 4.6 token volumes need to be revised upward by approximately 30%.

Model pool notes:

Opus 4.8, 4.7, and 4.6 share one rate limit bucket.
Sonnet 5 and Sonnet 4.6 likely share a pool — not formally confirmed yet; check your dashboard.
Sending traffic to multiple model versions within the same pool draws from one shared bucket, not separate ones.

Gemini 3.5 Flash scaled tier note: Gemini 3.5 Flash is GA and is Google’s current flagship. Scaled tier figures for 3.5 Flash are not yet published in public documentation. The 3.1 Flash-Lite figures shown above remain the last confirmed published numbers.

Cerebras & SambaNova

Both specialize in custom silicon for inference speed.

Cerebras

Hardware: Wafer-Scale Engine (WSE-3). Fastest published inference: approximately 2,100 tokens/second on Llama 4 Scout.

Tier	RPM	TPM	TPD
Free	30	60,000	1,000,000
Paid	Higher (contact sales)	Higher	Higher

Hosts Llama 4 Scout and other open models. The speed advantage is real: tasks that take 30 seconds on GPU-based providers finish in under 5 seconds on the WSE-3. Paid tier limits are not publicly documented.

SambaNova

Hardware: Custom RDU (Reconfigurable Dataflow Unit). Best time-to-first-token (TTFT): approximately 0.2 seconds.

Tier	RPM	Notes
Free	10–30 (varies by model)	Hosts up to Llama 4 Maverick for free

SambaNova offers Llama 4 Maverick (1M context) on the free tier — most other free-tier providers cap out at smaller models. TPM and TPD limits are not publicly documented.

Audio Models — ASH & ASD

Groq and Fireworks both publish audio-specific limits.

Provider / Model	RPM	RPD	ASH	ASD	Audio min/min
Groq (Free) — Whisper Large v3	20	2,000	7,200	28,800	—
Groq (Free) — Whisper Large v3 Turbo	20	2,000	7,200	28,800	—
Fireworks AI — Whisper v3-large	—	—	—	—	200
Fireworks AI — Whisper v3-turbo	—	—	—	—	400

Groq: 7,200 ASH = 2 hours of audio per hour of wall time. 28,800 ASD = 8 hours per day. Adequate for a podcast transcription pipeline on the free tier.

Fireworks: 200 min/min for Whisper v3-large, 400 min/min for v3-turbo. Concurrent streaming capped at 10 connections.

Cloud Aggregators — Azure AI & AWS Bedrock

Both use configurable per-deployment limits, not fixed tiers. The shared ratio is 6 RPM per 1,000 TPM.

Azure AI (Microsoft)

Deployment Type	How Limits Work
Pay-as-you-go (Standard)	TPM quota per model per region, auto-scales with usage
Provisioned (PTU)	Reserved throughput units, no per-request limits
Global/Data Zone	Higher default quotas, multi-region routing

Multi-model: admins can select GPT-5.5, GPT-5.4, Claude Opus 4.8/Sonnet 5, or Gemini 3.1 Pro (verify availability per region — not all models are deployed globally; Sonnet 5 and Gemini 3.5 Flash availability on Azure is not yet confirmed). Azure AI Quotas & Limits

AWS Bedrock

Hosts Claude (Opus 4.8, Sonnet 5, Haiku 4.5), Llama 4, Mistral, and Nova models with per-model, per-region quotas. Claude Fable 5 availability on Bedrock is being restored following the July 1 global redeployment — check the Bedrock model catalog for current status. Default quotas vary by model and can be increased via AWS Service Quotas console. Provisioned Throughput deployments remove rate limits entirely. AWS Bedrock Quotas

More Providers — Perplexity, Alibaba (Qwen), Moonshot (Kimi)

Provider / Model	RPM (T0)	RPM (T1)	RPM (T3+)	TPM	TPD (T0)
Perplexity — Sonar Pro	50	150	1,000	—	—
Perplexity — Sonar	50	150	1,000	—	—
Perplexity — Deep Research	5	10	40	—	—
Alibaba (Qwen) — Qwen3.7-Max*	—	600*	600*	1M*	—
Alibaba (Qwen) — Qwen3.5 Plus	—	15,000	30,000	5M	—
Alibaba (Qwen) — Qwen3.5 Flash	—	15,000	30,000	10M	—
Moonshot — Kimi K2.5	3	200	5,000	—	1.5M

*Qwen3.7-Max (May 20) limits not yet formally published. Figures shown are Qwen3 Max limits carried forward as a provisional floor.

Perplexity tiers: T0 = new account, T1 = $50+, T3 = $500+. Alibaba limits shown for international (Singapore) deployment; Beijing deployment is higher (Qwen3 Max at 30K RPM — Qwen3.7-Max Beijing limits not yet confirmed). Moonshot T1 = $10+ cumulative recharge; T0 has 1.5M TPD cap, T1+ is unlimited.

Standout: Alibaba’s Qwen3.5 Flash at 30K RPM / 10M TPM (Beijing) remains the highest confirmed throughput of any provider in this post. Moonshot’s T5 tier (10K RPM, 1,000 concurrent connections, $3,000+ recharge) is competitive at the high end.

Provider Notes

OpenAI: GPT-5.5 and GPT-5.4 share identical rate limit profiles across all tiers. GPT-5.4 Nano at Tier 5 (180M TPM, 30,000 RPM) is the throughput ceiling for any published provider. GPT-5.5 Instant is a ChatGPT product tier — not accessible via the API under that name; API access goes through the GPT-5.5 model identifier. GPT-5.6 Sol (previewed June 27) is in limited early access for trusted partners only — not generally available, not represented in rate limit tables. GPT-5.4-Cyber is available only to vetted security teams; rate limits not disclosed. GPT-5.2 is being phased out. GPT-5.3 Codex remains available for coding workloads. Batch API offers 50% discount with higher queue limits. Assistants API deprecates August 2026; replaced by Responses API. Docs
Anthropic: Opus 4.8 is the current flagship (May 2026). Claude Sonnet 5 (June 30, 2026) is the current balanced tier — its tokenizer emits approximately 30% more tokens per request than Sonnet 4.6 for identical text, which affects TPM consumption. Introductory pricing ($2/$10 per M) runs through August 31, 2026. Opus 4.8, 4.7, and 4.6 share a combined rate limit pool. Sonnet 5 and Sonnet 4.6 likely share a pool — not yet formally documented. Claude Fable 5 was restored globally July 1 after a brief export-control suspension; rate limit figures for Fable 5 are not yet published. Claude Mythos 5 is restricted to vetted US organizations via Project Glasswing; not generally available. Cached input tokens don’t count toward ITPM on current 4.x and 5.x models. No free tier; lowest entry is $5. Claude 3 Haiku retired April 2026. Docs
Google: Static rate limit tables removed from public docs in Q1 2026. Actual limits only visible in AI Studio dashboard (aistudio.google.com/rate-limit). Gemini 3.5 Flash is now GA (released May 19 at Google I/O 2026) and is Google’s current flagship — approximately 4x faster output than other frontier models, leads on agentic and coding benchmarks, includes computer use capability. Rate limit figures for Gemini 3.5 Flash are not yet published in public documentation. Gemini 3.5 Pro remains in limited preview (2M context, Deep Think). Gemini Omni rate limit figures also not yet published. Gemini 3.1 Pro remains available as the previous-generation Pro model; Gemini 2.5 Pro/Flash are previous-generation models no longer recommended for new deployments. Gemini 2.0 Flash/Flash-Lite shut down June 1, 2026 — those endpoints no longer accept requests. Docs
Groq: Free tier publishes exact numbers (30 RPM, 6K–30K TPM depending on model). Daily token budgets (TPD) alongside TPM. Cached tokens don’t count. Hosts Llama 4 Scout and other open models on free tier. Runs on custom LPU hardware. Docs
xAI: Current flagship is Grok 4.3 (April 30, 2026; 1M context; $1.25/$2.50 per M input/output). grok-build-0.1 is its agentic coding model. 5-tier structure ($0/$50/$250/$1K/$5K thresholds) but numerical RPM/TPM only visible in xAI Console. Docs
Mistral: European data residency (GDPR). 5-tier structure (Free through Tier 4 at $500+). Limits enforced per RPS (not RPM), TPM, and tokens/month. Actual numbers require login at admin.mistral.ai. Docs
DeepSeek: Fully dynamic concurrency limits based on server load. No published RPM/TPM. V4-Flash and V4-Pro are the current models. V3.2 aliases deprecate July 24, 2026 — two weeks out. Update any hardcoded model references now. V4-Flash pricing: $0.14/M input (cache miss), $0.028/M (cache hit), $0.28/M output. Docs
Cerebras: Fastest inference (approximately 2,100 tok/s on Llama 4 Scout). Free tier: 30 RPM, 60K TPM, 1M TPD. Paid tier limits not publicly documented. Custom WSE-3 silicon. Docs
SambaNova: Best TTFT (approximately 0.2s). Free tier: 10–30 RPM depending on model size. Hosts up to Llama 4 Maverick for free. TPM and TPD limits not publicly documented. Custom RDU hardware. Docs
Together AI: Dynamic rate limits since January 2026. No fixed tiers or published numbers. Limits grow with sustained usage and are returned in API response headers. Docs
Fireworks AI: Dynamic ceiling up to 6,000 RPM (soft limit starts approximately 1 RPS, doubles hourly). On-demand GPU deployments remove limits. Spending tier caps: $50–$50K/month by tier. Docs
Cohere: Command A+ (May 20, Apache 2.0, 218B MoE / 25B active; also runs on 1x B200) is the flagship. Self-hosting is viable on 2x H100 or 1x B200 and removes API rate limits entirely for teams with the hardware. API rate limit figures for Command A+ not yet published — 500 RPM (production chat) shown in tables is the Command R+ figure. Trial keys: 1,000 calls/month. Docs
Perplexity: 6 tiers (T0–T5) based on cumulative spend. Leaky bucket algorithm. Deep Research model has very low limits (5–100 RPM). Agent API: 50–2,000 RPM by tier. Docs
Alibaba (Qwen): Qwen3.7-Max (May 20) is the current proprietary flagship (1M context; $2.50/$7.50 per M); published rate limits not yet confirmed. Open-weight family is now Qwen3.6 (Apache 2.0 — e.g., Qwen3.6-35B-A3B, Qwen3.6-27B). Region-specific limits apply — Beijing deployment is more generous than Singapore/Global. Qwen3.5 Flash at 30K RPM / 10M TPM (Beijing) is the highest confirmed throughput of any provider listed. Docs
Moonshot (Kimi): 6 tiers (T0–T5). T0 ($1 recharge): 3 RPM, 1 concurrent, 1.5M TPD. T5 ($3,000): 10K RPM, 1,000 concurrent, 5M TPM, unlimited TPD. Automatic 75% caching discount applied without opt-in. Docs
AWS Bedrock: Hosts Claude (Opus 4.8, Sonnet 5, Haiku 4.5), Llama 4, Mistral with per-model per-region quotas. Claude Fable 5 availability on Bedrock being restored following July 1 global redeployment — check the Bedrock model catalog. 6 RPM per 1K TPM ratio. Provisioned Throughput removes limits. Docs
Azure AI: Hosts GPT-5.5, GPT-5.4, Claude Opus 4.8/Sonnet 5, and Gemini 3.1 Pro across regions (verify model and region availability before planning deployments — not all models are deployed globally). PTU deployments remove per-request limits. Docs
NVIDIA NIM: Hosted API is for prototyping (40 RPM). Self-hosted NIM containers have no limits. Docs

Tips for managing rate limits

For a deep dive on token pricing, cost optimization strategies, caching mechanics, model routing architectures, and production cost modeling, see LLM Token Costs and Efficiency. This section covers rate limit management specifically.

Handling 429s

Use exponential backoff when you hit 429s. The Anthropic SDK, OpenAI SDK, and most third-party clients handle this automatically. Don’t build your own retry loop unless you need custom jitter or circuit-breaking behavior.
Check response headers before you hit the wall. Together AI and Fireworks return your current rate limit state in every API response. DeepSeek adjusts concurrency limits dynamically based on server load. Reading these headers in production enables proactive throttling rather than reactive retries.

Sonnet 5 tokenizer: revise your throughput math

Claude Sonnet 5’s tokenizer emits approximately 30% more tokens per request than Sonnet 4.6 for identical prompts. If you sized your TPM headroom against Sonnet 4.6 traffic, those calculations are stale. A workload that consumed 20K ITPM on Sonnet 4.6 will consume approximately 26K ITPM on Sonnet 5 with the same inputs. Recheck your limits against actual Sonnet 5 token counts before migrating workloads, particularly at Tier 1 where ITPM is already constrained at 30K.

Caching for throughput (not just cost)

Prompt caching has a rate limit benefit separate from its cost benefit. The cost savings are covered in the token costs post. The rate limit implications:

Anthropic: Cached input tokens don’t count toward ITPM limits at all on Claude 4.x and 5.x models. With 80% cache hit rate, a 2M ITPM limit effectively handles approximately 10M total input tokens per minute. This is a throughput multiplier, not just a cost reduction. Given Sonnet 5’s higher per-token counts, caching is more important for staying within ITPM limits than it was on Sonnet 4.6.
Groq: Cached tokens don’t count toward TPM/TPD limits on the free tier, effectively expanding your daily token budget.
Moonshot (Kimi): Automatic 75% caching discount applied without opt-in. No cache management required.

Batch APIs: higher queue limits

OpenAI, Anthropic, and Google all offer batch endpoints with queue limits that far exceed real-time TPM. OpenAI’s GPT-5.5 gets 1.5M batch queue tokens at Tier 1 vs. 500K real-time TPM. For workloads that don’t need sub-second responses, batch endpoints let you move more tokens through tighter rate limits. See batch API cost savings for pricing details.

Model routing for rate limit distribution

Routing requests to different models by complexity distributes rate limit pressure across multiple buckets instead of concentrating it on your most constrained tier. A three-tier architecture (budget for triage, mid-tier for generation, flagship for reasoning) also cuts costs 60–85%. Full routing architecture with cost modeling is in the token costs post.

Custom silicon for fewer concurrent connections

Cerebras (approximately 2,100 tok/s) and SambaNova (approximately 0.2s TTFT) are faster than GPU-based providers by a large margin. A task that requires 10 parallel GPU-based API calls to meet a latency SLA might need 2–3 calls on Cerebras. Fewer concurrent calls means less rate limit pressure per unit of output.

Self-hosting as an escape hatch

Cohere’s Command A+ (Apache 2.0, 218B MoE / 25B active) runs on 2x H100 or a single B200. For teams already operating that hardware, self-hosting removes API rate limits entirely. The economics only work at scale — hardware cost needs to be weighed against API spend — but it’s a realistic option for a broader tier of teams than it was a year ago. The same applies to Llama 4, Devstral 2, Mistral Large 3, and other open-weight models.

Watch for shared pools and hidden constraints

Anthropic model pools: Opus 4.8, 4.7, and 4.6 share one rate limit bucket. Sonnet 5 and Sonnet 4.6 likely share another — not yet formally documented, check your dashboard. Sending traffic to multiple model versions within the same pool draws from one shared bucket.
Google’s invisible limits: With static rate limit tables removed from public docs, you must check AI Studio (aistudio.google.com/rate-limit) to see your actual per-project limits. Gemini 3.5 Flash is now GA but has no published rate limit figures. Third-party figures in this post for 3.1 Pro/Flash-Lite are the last confirmed published numbers and may be stale for current deployments.
DeepSeek’s dynamic ceiling and imminent deprecation: No published RPM/TPM means no guaranteed minimum. Under heavy load, effective concurrency can drop without warning. The July 24 V3.2 alias deprecation is two weeks out — audit your model endpoint references now. Build fallback routing to a second provider if DeepSeek availability is critical to your pipeline.
New model lag: Qwen3.7-Max and Command A+ both launched May 20, 2026. Gemini 3.5 Flash and Gemini Omni launched at Google I/O 2026. Claude Fable 5 was restored July 1. None of these models have formal rate limit tables published. Figures in this post for Qwen3.7-Max and Command A+ are provisional floors based on predecessor models. Fable 5 is absent from tables entirely until Anthropic publishes figures.