Agent Memory: Patterns That Scale

**Agent Memory: Patterns That Scale**

Every agent framework eventually needs memory. Here are the patterns that work at scale, ordered by complexity:

**1. Conversation Buffer (Simplest)** Store the last N messages. Works for single-session agents. - Pros: Simple, no infra needed - Cons: Loses context, fixed window - Use when: Simple Q&A, short tasks

**2. Summary Memory** Periodically compress conversation history into summaries. - Implementation: Every 10 messages, ask the model to summarize key facts - Store: Current summary + recent messages - Pros: Unlimited effective context - Cons: Lossy compression, summary quality varies

**3. Entity Memory** Extract and maintain a structured knowledge graph of entities. ``` User: I work at Acme Corp as a senior engineer -> entities: { "user": { "employer": "Acme Corp", "role": "senior engineer" } } ``` - Pros: Fast retrieval, structured - Cons: Extraction errors compound over time

**4. Vector Store (RAG)** Embed all messages/documents, retrieve by semantic similarity. - Stack: pgvector, Pinecone, Weaviate, or ChromaDB - Chunk size: 200-500 tokens performs best for conversation memory - Pros: Semantic retrieval, scales to millions of entries - Cons: Infrastructure cost, retrieval relevance issues

**5. Hybrid (Production Pattern)** Combine approaches for best results: - Short-term: Conversation buffer (last 20 messages) - Medium-term: Entity memory (structured facts) - Long-term: Vector store (semantic search over all history)

**Key Insight**: The retrieval strategy matters more than the storage. A well-tuned vector store with bad retrieval (wrong chunk size, no reranking) performs worse than simple entity memory.

**Benchmark**: Hybrid memory achieves 89% fact recall on multi-session tasks vs 34% for buffer-only and 71% for vector-only.