Agent memory vs RAG: decide what should be remembered
Separate task state, user preferences and source documents, then choose storage, retrieval and deletion rules for each type of information.
A useful assistant may need to continue an unfinished task, remember a user's preferences and consult a changing knowledge base. Those are different requirements even when they share storage or search components. This source-reviewed guide uses a fictional product-support assistant to explain our design approach. It does not compare the measured performance of memory products.
Separate what happened from what is true now
LangChain's memory overview distinguishes conversation-scoped state from information retained across conversations. Its retrieval guide describes fetching external knowledge when it is needed for generation. These ideas can work together: an application can retrieve stored memories, and a conversation can contain retrieved documents. Memory and RAG are not mutually exclusive product categories.
For the support assistant, the current ticket number is task state. A user's chosen reply language is a preference. The current warranty policy is a reference document. A past conversation saying that a warranty lasts longer is neither a reliable policy update nor permission to change the official source. Decide the authority of each record before choosing a database.
Give each record an owner and a lifetime
Our suggested worksheet starts with who may see the information, who can correct it and when it should stop influencing an answer. Avoid an undifferentiated collection of everything the assistant has encountered. A convenient search result should not gain the authority of a verified account record merely because it was retrieved first.
| Information | Suggested handling | Authority |
|---|---|---|
| Pending support draft | Task state with explicit completion status | Current workflow |
| Preferred reply language | Editable user preference | User confirmation |
| Warranty policy | Versioned reference with effective date | Published policy owner |
| Tentative account match | Temporary evidence requiring confirmation | Unresolved |
| Access credential | Dedicated secret storage, excluded from retrieval | Account authorization |
Design the write path as carefully as retrieval
Suppose a customer says, 'Please use Spanish for this reply.' Our proposed behavior records the choice for this ticket unless the user clearly asks for an ongoing preference. The assistant can offer to save that preference, but it should not silently turn a one-time instruction into a permanent profile fact.
Store enough provenance to explain a remembered fact: its source, owner, scope, creation time and later corrections. Make replacement explicit. If the user changes the preference, the next answer should not retrieve both old and new values and improvise a compromise. A simple structured field can be easier to manage than semantic search for this case.
Retrieve within the correct account and source version
When answering a warranty question, first identify the product and applicable policy version, then retrieve supporting passages. Show the source used and state when the available documents do not resolve the question. Our suggested answer contract distinguishes a cited policy statement from a suggestion to contact support.
Apply account and tenant restrictions before returning candidate records to the model. Do not use the model as the final filter for information it should never receive. Keep a practical deletion procedure for original records, search indexes and retained copies, with the retention limits explained to the user. Deleting one chat message does not by itself prove that every derived record disappeared.
Test the specific behavior you want to remember
Build a small acceptance set around the fictional assistant. Continue an unfinished ticket after a restart. Change the reply language. Replace a policy with a newer version. Ask a question with no supporting document. Repeat those tasks as a different account. Inspect the stored records as well as the answer.
Choose the smallest design that passes these checks. Add cross-session memory when remembering approved preferences improves the task. Add retrieval when answers require external evidence. Add both when the job calls for both, with independent ownership and deletion rules. The architecture should make a wrong memory easy to find and correct, rather than merely making more information searchable.
Sources & further reading
Official sources support the specific product and protocol facts in this guide. The selection process and examples are editorial recommendations, not a claim that we tested the listed tools.