Semantic Memory
A type of agent memory that stores general facts, domain knowledge, embeddings of documents, and distilled insights—without temporal binding—enabling knowledge-base-style retrieval for any task.
Semantic memory corresponds roughly to a knowledge base or long-term fact store. Unlike episodic memory (what happened when), semantic memory holds timeless or slowly-changing facts: product documentation, domain ontologies, company policies, research findings. It is the primary target of RAG pipelines.
Implementation typically uses a vector database (Chroma, FAISS, Pinecone) that stores embeddings of text chunks. At query time, the agent embeds its query and retrieves the nearest neighbors, then re-ranks them. The resulting chunks are injected into the context window to ground the model's response.
Maintenance of semantic memory is an ongoing process: new documents must be ingested and indexed, stale documents updated or removed, and the embedding model periodically upgraded when a better model becomes available (which requires re-embedding the entire corpus). Tools like LlamaIndex and LangChain provide pipelines for this maintenance work.