Office Hours — Should ontology-driven approaches replace traditional prompting and RAG for AI coding tasks?
A daily developer question about AI/LLMs, answered with a direct, opinionated take.
Should ontology-driven approaches replace traditional prompting and RAG for AI coding tasks?
No. Ontologies are a tool for specific problems, not a replacement for prompting and RAG. The question frames this as an either/or when real production systems use a mix.
What ontologies actually do
An ontology is a formalized representation of domain knowledge, structured as entities, relationships, and rules. For coding tasks, this might mean encoding your codebase’s architecture (services, dependencies, data models) as a queryable graph instead of shoving it all into context.
The appeal is obvious: instead of asking an LLM to hallucinate the shape of your system from 100 scattered code snippets in its context window, you give it a precise map. Less ambiguity. Fewer wrong assumptions about what exists where.
Cursor’s agent swarm (July 2026 reporting) validates part of this intuition. Separating planning from execution, with cheaper models handling implementation once a frontier model has reasoned about the task structure, shows that explicitness about what to do helps. But that’s different from saying “use an ontology instead of RAG.” Cursor still retrieves context. It’s just better orchestrated.
Where ontologies fail for coding
The friction is that building and maintaining an ontology is expensive. Your codebase changes constantly. Refactorings, renamed modules, new services, deprecated endpoints. An ontology becomes stale fast, and a stale ontology is worse than fuzzy retrieval because it lies with confidence.
RAG doesn’t have this problem. It’s lazy. You index your actual code, your tests pass or fail, and the retrieval happens against live source. When code changes, the index adapts naturally. When you ask “where’s the payment handler,” vector search finds it even if nobody updated a formal schema.
Ontologies also assume your domain can be formalized cleanly. For a well-structured microservice architecture with clear APIs and versioning, maybe. For a monolith with tangled dependencies and implicit contracts? Ontologies force you to answer questions about your system that you’ve been dodging for three years. That’s either painful enlightenment or a dead-end.
The real workflow
What’s actually working in production is messier. A few concrete patterns:
Hybrid retrieval for coding: Use RAG to pull relevant code snippets, but add lightweight metadata tagging (this file is a route handler, this is a database schema, this is a test fixture). When the LLM asks “show me all database writes,” you can filter the retrieval by that tag instead of just doing similarity search. You’re not building a full ontology, just adding semantic hints to the retrieval layer.
Structured prompts with implicit ontologies: Some teams encode architectural knowledge in the system prompt itself, essentially embedding a lightweight ontology as instruction. “Our auth layer is always in services/auth. Database models live in models/. HTTP routes are in routes/…” This is a flattened ontology, and it works because it’s versioned with your prompt and updated when architecture changes.
Agents with tool discovery constraints: Gemini’s Managed Agents (July 2026) and Claude Code both let you specify which tools an agent can call and in what order. That’s a micro-ontology—you’re saying “before you call the deployment tool, you must run tests.” You’re not replacing RAG; you’re adding guardrails to what the agent can do with retrieved context.
Agentic RAG with fallback: When RAG fails (can’t find the right context), some teams use a secondary step where the agent reasons about the architecture and makes an educated guess. That’s ontology-like reasoning, but only on demand and only when pure retrieval failed. You save the cost and maintenance of a full formal model.
The case for ontology-first only works if
Ontologies become the right default when:
You have a stable, well-documented system architecture that rarely changes structurally. Financial institutions with tightly governed schema changes, or enterprise platforms with formal architectural review boards.
Your coding task is narrowly scoped, like “generate a new microservice that follows our pattern.” You can give the LLM a template ontology and say “fill in the gaps.” The ontology constrains the solution space.
You’re willing to invest a team in maintaining the ontology. This is implicit. Most teams won’t do it unless governance forces them.
What actually matters
The recent Daily Signal coverage (July 27) notes that agents are drowning in tool proliferation with no mechanism to gracefully decline irrelevant ones. That’s a tool discovery problem, which an ontology could solve. But simpler solutions like filtering tool lists by task type, or using multi-step planning to narrow the tool set, often work first.
Poolside’s Laguna model (July 23 reporting) solved a 50-year-old math problem by teaching small models self-correction and persistence, not by feeding them better structured knowledge. That’s a reminder that how you orchestrate reasoning often matters more than how formally you represent domain knowledge.
Bottom line: Layer ontologies on top of RAG and prompting when your codebase is stable enough to maintain formal schema, but don’t tear out working RAG systems to chase the ontology ideal. Hybrid retrieval with semantic hints and structured prompts get you 80% of the benefit at 20% of the cost.
Question via Hacker News