Office Hours — What's the best way to replace RAG pipelines with a filesystem interface for AI agents?

What’s the best way to replace RAG pipelines with a filesystem interface for AI agents?

Don’t. RAG and filesystem interfaces solve different problems, and conflating them usually means you’re optimizing for the wrong constraint.

The Confusion

The appeal is obvious. RAG adds latency, costs money per query, and introduces a failure point (retriever doesn’t find what you need). A filesystem interface is immediate, costs nothing, and feels simpler. If your agent can just read /data/latest_report.txt instead of hitting a vector database, why wouldn’t you?

Because RAG exists precisely because filesystems don’t scale to the retrieval problem you’re actually trying to solve.

What Filesystem Access Solves

Direct filesystem access is legitimately useful for agents that need to browse structured project layouts, access configuration files, or iterate through logically organized directories. GitHub Copilot reading your codebase works this way. Devin examining a repo structure to understand the architecture before writing code works this way. If your agent’s task is “implement feature X in this specific project,” filesystem access with semantic understanding of structure is the right move.

The strength: the agent sees the actual structure of information as it exists. It doesn’t need a retriever guessing what’s relevant. It can explore.

What Filesystem Access Breaks

The moment you need relevance ranking, it falls apart.

Say you have 50,000 customer support tickets and your agent needs to answer “what’s the common complaint pattern this week?” Filesystem access means either the agent reads all 50,000 files (token budget explodes), or it reads a sample and hopes it’s representative (lossy and unreliable). A hybrid search with re-ranking—as documented in the May 12 Daily Signal—gives you the top 10 most relevant tickets ranked by semantic similarity and recency.

Filesystem access also doesn’t scale to multi-source retrieval. If your knowledge lives across three databases, two API endpoints, and a CRM, you’re asking the agent to orchestrate connections to each one. RAG abstracts that heterogeneity into a single query interface.

The Actual Hybrid Approach

What actually works in production is layered: use filesystem access for structure and code, RAG for knowledge retrieval.

# Agent can directly explore code structure
agent.read_file("/src/api/routes.py")

# But uses RAG for domain knowledge
relevant_docs = retriever.search(
    "How do we handle pagination in list endpoints?",
    top_k=5,
    rerank=True
)

# Combine both contexts
context = {
    "codebase_structure": file_tree,
    "relevant_patterns": relevant_docs,
    "current_file": agent.read_file(target)
}

This works because each interface solves what it’s good at. The filesystem gives the agent ground truth about actual code organization. RAG gives it semantic understanding of patterns and conventions.

When Filesystem Only Works

If your data is inherently hierarchical, small enough to fit in context, and doesn’t require relevance ranking, filesystem access alone is fine. Private documentation that’s organized by purpose. Configuration files. A single codebase. Logs from a single service. These are fine to traverse directly.

But the moment you’re asking “find me X” rather than “show me Y,” RAG earns its keep. The hybrid search and re-ranking approach mentioned in the Daily Signal (May 12) represents current production best practice for a reason: it works better than either approach alone.

The Real Cost Analysis

Filesystem access: free but high token cost, latency creep as you read more files, degrading agent focus.

RAG: $0.001–0.01 per query depending on embedding model and index size, but constant token cost, better relevance, faster agent decision-making.

On a 10,000-call/day agent workload, good RAG costs $10–100/day. Filesystem access costs $50–200/day in tokens because the agent has to read more context to get the same confidence level. RAG usually wins on both cost and reliability.

Bottom line: Replace RAG with filesystem access only if your data is small, hierarchical, and doesn’t require relevance ranking. Otherwise, layer them: filesystem for code structure and navigation, RAG for semantic search across larger knowledge bases.

Question via Hacker News