The model context window is not durable memory. It is the input to one inference call. If an agent needs information after that call, the application must store it and decide when to provide it again.
This sounds like a storage problem. It is mainly a selection and governance problem. Stored information can improve continuity, but incorrect or stale information can affect every later decision that retrieves it.
A useful memory design answers five questions: What can the system store? Who owns it? How does it retrieve it? When does it expire? How can a user correct or delete it?
Separate context, run state, and long-term memory
For product design, it helps to separate three systems that are often called memory.
Working context contains the instructions, messages, retrieved records, tool definitions, and observations available to the current model call. The application rebuilds this context on each turn. It can include information from memory, but the context itself ends with the call.
Run state records exact facts about current work. It can include completed actions, artifacts, budgets, errors, pending approvals, and the current status. Long-running or resumable agents need this state to survive a worker restart. Keep exact identifiers and state transitions in structured records. Do not rely on a natural-language summary to preserve them.
Long-term memory stores information for later conversations or runs. Research often divides this information into semantic, episodic, and procedural memory. Semantic memory contains facts. Episodic memory contains past experiences or events. Procedural memory contains instructions or learned ways of performing a task.
These categories describe what the information means. Context, run state, and long-term storage describe where and when the application uses it. The two views can work together. A verified customer preference is a semantic fact stored for later use. A successful past task may become an episodic record. A reviewed procedure may become a versioned instruction.
Execution checkpoints are application state, not memory in the usual research sense. Organization policy should also remain in its authoritative system. An agent may retrieve a policy, but a conversation should not update it without review.
Decide what the agent may store
Every memory write can affect a future response or action. The write policy should define eligible information, ownership, source, scope, retention, and update rules.
Stable user preferences may be useful. Temporary project facts need an expiry. Sensitive information may require a specific purpose and user control. Claims about another person may not be appropriate to store. High-consequence facts should come from authoritative records when those records exist.
A production memory record can include the tenant and user scope, source, creation time, valid time, type, verification status, and expiry. Valid time matters because creation time does not say when a fact applies. A policy written today may take effect next month. A role recorded last year may no longer be active.
Avoid using an undefined model confidence score as authority. Language models do not always report calibrated confidence. Record whether the source was verified, which system owns it, and how the application derived any numerical score.
The application can write memory during the current interaction or process it later. A hot-path write becomes available at once, but a mistaken interpretation can affect the next turn. Background processing adds delay but can compare more evidence, remove duplicates, and apply review.
Conflicting facts need an explicit rule. The system can prefer the current authoritative source, keep the earlier version for audit, and mark which record is active. It should not merge statements that apply to different dates or scopes.
Procedures need stricter review than preferences. A successful trace can contain shortcuts, leaked information, or behavior that exploited a weak grader. Before the system promotes a repeated pattern into shared instructions, test it on held-out tasks and require an accountable review.
Retrieval determines what affects the next response
Retrieval should apply access controls before relevance ranking. Tenant, user, and permission boundaries are mandatory. A relevance score must never allow a record from another account to enter context.
Other filters depend on the task. Type, date, and project filters can improve precision, but a strict recency rule may hide an older fact that still applies. Ranking can use source authority, verification status, valid time, recency, lexical match, semantic similarity, and task relevance.
Vector search is one option, not the definition of memory retrieval. Structured lookup works well for known identifiers and fields. Lexical search preserves exact names and terms. Semantic search finds related language. Hybrid systems combine these methods and apply filters before or after candidate retrieval according to the query.
Irrelevant, conflicting, or badly placed context can reduce accuracy. Research on long-context models has shown that position affects how well models use information. Loading every stored item is not a reliable retrieval strategy.
One practical pattern is progressive retrieval. The first query returns compact candidates with source and scope. The agent or application then loads the full record needed for the decision. This can reduce context use and unnecessary exposure of private content. It is not required for every task.
The system should abstain when it finds no valid memory. A weak semantic match should not become a fact because the agent prefers a complete answer. For consequential decisions, the agent can ask the user or query the authoritative source.
Protect memory from persistent prompt injection
Memory can preserve hostile instructions as well as useful facts. An email or webpage may contain text that tells a future agent to ignore policy, disclose data, or call a tool. If the application stores that text, the attack can affect later runs.
Marking retrieved text as untrusted helps the model interpret it, but a label does not enforce security. The harness must keep authorization, tool policy, and tenant isolation outside the model. A retrieved memory should not grant permissions or change the authority of the user who started the run.
Controls apply at several points. Validate the source and scope before writing. Preserve provenance in the record. Apply access controls during retrieval. Keep untrusted content separate from system instructions. Check policy again before any action. Monitor for unexpected memory writes and cross-tenant retrieval.
Memory poisoning tests should include false facts, hidden instructions, repeated attempts to overwrite verified records, and records that resemble another tenant's data. The test should inspect the stored item, retrieved context, model behavior, and final action. Testing only the final response can miss a poisoned record that will affect a later task.
Memory needs correction, expiry, and deletion
Memory controls affect product correctness as well as privacy. Users need a way to inspect and correct durable information that changes their experience. Organization administrators need controls over shared records. Temporary facts should expire when they no longer apply.
Deletion needs defined behavior across source records, indexes, caches, traces, and derived summaries. Lineage can connect a derived memory to its source. A tombstone can prevent an asynchronous index from restoring a deleted item. Backups expire under a documented schedule rather than changing at once. The retention design should state the schedule.
Preserve ownership metadata in every cache, index, and trace. A tenant filter in the main database does not protect an unscoped embedding cache or a debugging export. Test isolation at each point where memory data moves.
Store less when a defined future task does not need the information. This reduces stale behavior, retrieval noise, and the impact of unauthorized access. A general store with no clear purpose becomes difficult to govern and difficult to evaluate.
Organization knowledge and personal memory also need separate scopes. Company policy belongs to the organization and should retain its authoritative source and version. A user's writing preference belongs to that user. A current task observation may apply only to one run. The application should identify which source affected a decision.
Test each stage of the memory system
Memory evaluation should separate writing, storage, retrieval, use, and deletion. One total score cannot identify which stage failed.
Writing tests measure whether the system stores useful facts and rejects ineligible ones. Retrieval tests measure whether it returns the right record under the right identity and time. Use tests measure whether the agent applies the record to the current task. Deletion tests confirm that the item no longer appears in active stores, indexes, and caches according to policy.
Suites should cover information extraction, multi-session reasoning, temporal reasoning, knowledge updates, conflict resolution, and abstention. Include wrong-user near matches, revoked facts, unverified sources, and requests to forget. Test both positive cases and cases where the system should store or retrieve nothing.
Version the write policy, retrieval and index configuration, embedding model where used, ranking logic, and context placement. A model update can change how strongly a retrieved statement affects behavior. A retrieval change can improve recall while increasing boundary violations.
An agent memory system stores selected information for future use and preserves the ownership, source, time, and authority needed to use it correctly. The system should store information only when it improves a defined task. It should also support correction, deletion, and safe abstention when no valid memory exists.