The Stack — Cursor

Cursor is an AI-first code editor built on top of VS Code that uses frontier models to autocomplete, explain, and rewrite code at the file and repository level.

What It Is

Cursor is a fork of VS Code that deeply integrates large language models into the editing experience — not as a plugin, but as a first-class interface primitive. It’s used primarily by individual developers and engineering teams who want AI assistance that understands their entire codebase, not just the current file. The product competes with GitHub Copilot but differentiates on depth of context and multi-file editing capabilities.

The Architecture

Cursor’s most distinctive infrastructure choice is their approach to context construction. Rather than naively stuffing your current file into a prompt, Cursor appears to run a retrieval layer that indexes the entire repository using a combination of AST parsing, embeddings-based semantic search, and graph-based dependency resolution. When you trigger a completion or a chat, the system assembles a context window from multiple sources: the current file, relevant imports, recently edited files, and semantically similar code chunks. This is RAG, but purpose-built for code structure rather than document search.

For model routing, Cursor almost certainly runs a tiered inference strategy. Autocomplete (the inline ghost-text) needs sub-200ms latency to feel native, so that path likely uses a smaller, faster model — possibly a fine-tuned variant of Llama 4 or a custom model — served on their own infrastructure. The chat and “Composer” (multi-file edit) features have more latency tolerance and appear to route to frontier models: Cursor has publicly confirmed partnerships with Anthropic and OpenAI, and their “Max” tier explicitly lets users select Claude Opus 4.6 or GPT-5.4 for the heaviest tasks.

Their context window management is a real engineering problem at their scale. Repository-wide awareness means potentially millions of tokens of source code, but frontier model inference at that context length is expensive and slow. Cursor’s apparent solution is aggressive pre-filtering: they run a cheap relevance-scoring pass (likely a small embedding model) to select the top-K code chunks before hitting the expensive model. This keeps prompt lengths manageable without sacrificing the “it understands my whole codebase” experience.

On infrastructure, Cursor appears to run a hybrid hosting model. High-volume, latency-sensitive completions route to self-hosted or reserved capacity, while complex reasoning tasks fall back to the major API providers. This architecture lets them control cost and latency on the hot path while avoiding the capital expenditure of running frontier models themselves. Their “Usage-based” pricing tiers suggest they have visibility into per-request costs and are passing some of that complexity to users deliberately.

The Smart Decision

The smartest single architectural decision Cursor made is treating the editor state as a first-class data stream rather than a request/response API call. Most AI coding tools are stateless: you ask a question, you get an answer. Cursor’s diff-aware editing and multi-turn Composer sessions maintain a persistent representation of what’s been changed, what’s been accepted, and what the current intent is across multiple model calls.

This is meaningful because it transforms multi-file edits from a “generate and pray” operation into an iterative, inspectable process. Each model call has access to prior deltas, not just the starting state — which dramatically reduces the hallucination surface area on complex refactors. The practical result is that Cursor can do things like “rename this abstraction across 40 files” with reasonable reliability, where a stateless tool would generate 40 independent guesses. The engineering cost of this is significant (they essentially built a lightweight state machine around model outputs), but the product differentiation is durable.

The Tradeoff

Cursor’s deep VS Code fork is both their moat and their most significant liability. By forking rather than building a plugin, they can instrument the editor at a level no plugin can match — but they’ve committed to permanently tracking VS Code’s release cadence while also maintaining their own feature layer on top. Every upstream VS Code update is a potential merge conflict. Every new VS Code extension that users want may behave unexpectedly against Cursor’s modified internals.

This tradeoff becomes more expensive as the product matures. In 2024-2025, moving fast on a fork made sense. By 2026, they’re carrying meaningful technical debt from two years of divergence, and competitors building natively on the Language Server Protocol (LSP) or on cleaner abstractions are starting to close the feature gap. The fork gives them a speed advantage today and a maintenance tax indefinitely — a classic startup tradeoff that looks different at scale than it did at founding.

What You Can Steal

• Route by latency tolerance, not just capability. Don’t use the same model for inline autocomplete and complex reasoning. Define your latency SLA first, then pick the model that fits inside it — not the best model you can afford.

• Pre-filter context before hitting expensive models. Run a cheap embedding-based relevance pass to select top-K context chunks. This is especially important for any domain with large corpora (code, docs, legal, medical). The small model doing pre-filtering pays for itself many times over.

• Make model provider selection a user-facing control. Cursor’s explicit model picker (“use Opus 4.6 for this task”) reduces support burden, aligns cost with value, and trains users to think about capability tiers. It’s counterintuitive, but exposing complexity here builds trust.

• Treat inference state as a session, not a request. If your product involves iterative AI outputs (edits, drafts, plans), persist intermediate state between model calls. Downstream calls with delta-awareness produce dramatically better results than re-generating from scratch.

• Hybrid hosting isn’t all-or-nothing. You can self-host or reserve capacity for your high-volume cheap path while using API providers for low-volume expensive calls. Design your routing layer early so you can shift workloads as pricing and model quality change.