Office Hours — How do you build persistent memory systems for AI agents that work across multiple sessions and different deployment environments?

How do you build persistent memory systems for AI agents that work across multiple sessions and different deployment environments?

The Core Problem

Memory systems for agents aren’t just about storing data—they’re about storing the right data in the right format, fast enough to fit inside a context window, across sessions that might run on different hardware in different clouds. The naive approach (dump everything into a vector database and RAG it) collapses under real-world constraints: agents forget context mid-task, token costs spiral when you load 10,000 past interactions to find the three relevant ones, and when your agent moves from local Docker to a cloud environment, suddenly the database connection string breaks and your memory goes dark.

Most production agent systems fail on memory not because the retrieval is bad, but because they treat memory as an afterthought instead of a first-class architectural concern.

Three Layers: What to Remember

Stop thinking of memory as one thing. Effective agent systems separate memory by scope and lifetime.

Session memory (short-term, thread-scoped): This is the conversational thread—the immediate context the agent needs for the current task. In LangGraph or Claude Cowork, this maps directly to a thread ID. Costs pennies. Lives in fast storage (Redis, in-memory, or just in the function stack). Expires after 24-48 hours or after task completion.

Persistent memory (long-term, cross-session): Facts the agent learned that matter for future sessions—completed tasks, user preferences, discovered patterns, past decision outcomes. This grows slowly and should be intentionally curated, not auto-populated. Store this in a database with searchable schemas, not a vector store dump.

System memory (configuration, not data): The agent’s capabilities, tools, guardrails, and operational parameters. This rarely changes but absolutely must not be stored in the same table as conversation history, because when you deploy a new version of the agent, you want a clean slate for system config without losing learned facts from past sessions.

Concrete Pattern: Scope-Based Storage

Here’s a working architecture:

# Thread-scoped (session) memory
class SessionMemory:
    def __init__(self, thread_id: str):
        self.thread_id = thread_id
        self.context_buffer = []  # Last N messages/tool calls
        self.working_facts = {}   # Transient state during this task
    
    def add_turn(self, role: str, content: str):
        self.context_buffer.append({"role": role, "content": content})
        # Keep only last 20 turns to stay under ~100K tokens
        if len(self.context_buffer) > 20:
            self.context_buffer.pop(0)

# Cross-session memory
class PersistentMemory:
    def __init__(self, agent_id: str, storage_backend):
        self.agent_id = agent_id
        self.storage = storage_backend  # PostgreSQL, MongoDB, DynamoDB
    
    def record_fact(self, fact_type: str, key: str, value: dict, ttl_days: int = 90):
        """Record a learned fact with explicit type and TTL."""
        self.storage.insert({
            "agent_id": self.agent_id,
            "fact_type": fact_type,  # "user_preference", "completed_task", "error_pattern"
            "key": key,
            "value": value,
            "timestamp": now(),
            "expires_at": now() + timedelta(days=ttl_days)
        })
    
    def retrieve_relevant_facts(self, query: str, limit: int = 5):
        """Structured retrieval, not vector search."""
        # Query by fact_type and key prefix, not semantic search
        # Example: "user_preference:timezone" returns user's timezone
        return self.storage.query(
            agent_id=self.agent_id,
            key_prefix=query,
            limit=limit,
            order_by="timestamp DESC"
        )

The key insight: type and key the facts explicitly. A fact isn’t “the user said they live in PST”—it’s fact_type="user_preference", key="timezone", value={"tz": "PST"}. When the agent later needs to know the user’s timezone, you ask the database directly: retrieve_relevant_facts("user_preference:timezone"). No embedding model needed. No false positives. Fast.

Cross-Environment Persistence

When agents run in different environments (local dev, staging, production cloud), memory either breaks or requires tight coupling to a specific database endpoint.

Solution: abstract the storage backend. Define a simple interface and swap implementations:

class MemoryBackend(ABC):
    @abstractmethod
    def insert(self, record: dict) -> None: pass
    @abstractmethod
    def query(self, **filters) -> list: pass

class PostgresBackend(MemoryBackend):
    def __init__(self, connection_string: str):
        self.conn = psycopg2.connect(connection_string)

class LocalSQLiteBackend(MemoryBackend):
    def __init__(self, db_path: str):
        self.conn = sqlite3.connect(db_path)

# At startup, pick the backend based on environment
if os.getenv("DEPLOYMENT_ENV") == "local":
    memory = PersistentMemory("agent-1", LocalSQLiteBackend("./memory.db"))
else:
    memory = PersistentMemory("agent-1", PostgresBackend(os.getenv("DATABASE_URL")))

For cloud deployments across multiple availability zones or regions, use a managed database (PostgreSQL on RDS, DynamoDB, Firestore) with built-in replication. Avoid embedding the connection string in the agent code—inject it via environment variables or secrets management (AWS Secrets Manager, HashiCorp Vault).

The Token Budget Reality

Most agent memory systems fail because they load too much into the prompt. If an agent has 100 past sessions, and you naively prompt it with summaries of all of them, you’re burning 50K tokens before the agent even thinks.

Always compute context size upfront:

def prepare_agent_context(agent_id: str, thread_id: str, max_tokens: int = 50000):
    """Build context for this agent call, respecting token budget."""
    session_mem = SessionMemory.load(thread_id)
    persistent_mem = PersistentMemory(agent_id)
    
    # Session memory is mandatory (it's the current task)
    session_text = format_messages(session_mem.context_buffer)
    session_tokens = count_tokens(session_text)
    
    # Persistent memory is optional—only load what fits
    remaining_budget = max_tokens - session_tokens - 5000  # Reserve for output
    
    if remaining_budget > 0:
        relevant_facts = persistent_mem.retrieve_relevant_facts(
            # Query based on what the current task needs, not everything
            query="user_preference:*",
            limit=min(10, remaining_budget // 100)  # ~100 tokens per fact
        )
        facts_text = format_facts(relevant_facts)
    else:
        facts_text = ""  # No budget; skip persistent memory
    
    return session_text + facts_text

The rule: session memory is free (it’s context you already have), persistent memory is a luxury that only gets loaded if it fits the budget. If the agent is running on a cheap model with a 50K context window and you have 100K tokens of facts, you pick the 5 most relevant ones—not all of them.

Decay and Cleanup

Persistent memory grows unbounded unless you actively manage it. Set explicit TTLs (time-to-live) when recording facts. Facts older than 90 days about a task the user won’t repeat again? Gone. Facts about a user’s timezone? Never expire, or just refresh on confirmation.

-- Clean up expired facts daily
DELETE FROM agent_memory 
WHERE expires_at < NOW() 
  AND fact_type NOT IN ('user_preference', 'system_config');

What This Looks Like in Practice

You’re running Claude Opus 5 as an agent that processes customer support tickets. On the first ticket from a customer, the agent has only session memory (the ticket itself). It solves the ticket. Before

Question via Hacker News