The AI memory problem: Knowing what to keep and what to forget

Remembering is not the same as learning

In this article, memory means information an application stores and makes available to a model later. Saving a preference, retrieving a past decision, or appending a project summary does not, by itself, retrain the model. The application is changing the evidence the model sees. That makes memory a product and data-management problem as much as a model-capability problem.

This distinction changes what a founder should promise. An assistant can appear to know a customer better while simply receiving a growing profile before every response. If that profile contains an incorrect assumption, repeated exposure can make the mistake look remarkably consistent. Smooth personalization is therefore weak evidence that the system understands the person. The more useful question is whether the stored information is appropriate, current, and supported.

OpenAI's September 2025 engineering example on session memory describes a related tradeoff. Keeping only recent turns drops older material, while summarization can preserve selected context but lose detail or carry mistakes forward. Neither technique automatically establishes durable truth. Our view is that teams should treat a generated memory as a candidate record with an origin and a purpose, rather than promoting fluent text directly into an authoritative profile.

Primary source: OpenAI: Short-term memory management with sessions

Keep a working set, not a permanent monologue

A useful starting point is to separate three things: the original record, the currently accepted facts, and the context needed for today's task. A meeting transcript belongs in the first category. The agreed launch date belongs in the second. A request to draft a supplier email may need only a small selection from either. Combining all three into one endlessly expanding prompt makes them harder to inspect and update.

Anthropic's September 2025 context-engineering article describes retrieving information when needed through lightweight references, alongside techniques such as compaction and persistent notes. These are different ways to manage what reaches an inference call. The practical distinction is simple: information can remain available without being present in every response. Storage and attention are separate decisions.

For the launch assistant, the working set might contain the current target date, the latest supplier commitment, and the communication's intended audience. Older schedules can remain searchable when someone asks how the plan changed. They need not compete with the current schedule when drafting today's announcement. This is a proposed product pattern, not a universal architecture: some jobs genuinely require a long history. The point is to make that need explicit before paying to retrieve and interpret everything.

Primary source: Anthropic: Effective context engineering for AI agents

A useful fact can still belong in the wrong place

Consider the instruction, 'Keep this board memo under one page.' It is useful within the current task. Turning it into 'This user always wants short documents' changes its meaning. A future technical review may need ten pages. The memory system has not invented a preference from nothing; it has widened a local instruction until it becomes wrong.

Our suggested design gives remembered information an explicit scope. A personal writing preference, a team convention, a customer requirement, and a temporary project decision should not share one undifferentiated profile. Before a record is reused, the application should know whose work it applies to and which context made it relevant. Similar wording is not enough to establish that two records belong together.

This matters even within a single customer account. A consultant may serve several clients with conflicting terminology. A founder may discuss an experiment before deciding against it. A sentence beginning 'We could try' should not quietly become an established company policy. The writing step deserves scrutiny: what qualifies for persistence, what remains tentative, and what must be confirmed? An assistant that saves fewer, well-scoped facts can be more useful than one that extracts a confident profile from every exchange.

Let a remembered claim point back to evidence

When the assistant states that the launch is in February, a person should be able to discover why. Was the date confirmed by the founder, copied from a planning document, or inferred from a discussion? These origins deserve different treatment. A source reference does not prove the claim is correct, but it gives the team something concrete to inspect when the answer is challenged.

Google Cloud's Memory Bank documentation provides one implementation example: it distinguishes consolidated memory from historical revisions and supports source identifiers as labels. That separation makes a useful architectural idea visible. What the system currently uses and the sequence of changes that produced it need not be the same record. The documentation describes a particular service, not a requirement that every product use its storage model.

For a custom product, we would consider storing the source, when the claim was observed, the period it applies to, and what it replaces. Those fields answer different questions. A planning note written today might describe last quarter. Two disagreeing statements might concern different regions, rather than one being false. Preserving that distinction gives the assistant a chance to explain a change instead of flattening history into a single sentence. Where the evidence remains contradictory, uncertainty should survive the consolidation process.

Primary source: Google Cloud: Memory revisions

Freshness is more specific than a timestamp

A preference for British spelling may remain useful for years. A project's delivery date can change in a morning. Treating both as facts with the same expiry rule ignores why they were saved. Age is a signal about whether to recheck information; it is not a measurement of truth. Yesterday's speculation can still be less reliable than last month's signed-off plan.

One practical approach is to separate facts that are relatively stable from facts whose usefulness depends on current conditions. The latter may need confirmation from the system that owns them before an important action. The assistant can remember that an invoice exists while checking its present status in the billing record. Recalling a previous balance is a poor substitute for reading the current one.

Revalidation also needs a failure policy. If the current source cannot be reached, should the assistant show the last-known value with its date, ask a person, or pause the action? The answer depends on the consequence of being wrong. Writing a rough internal draft and making a customer commitment deserve different standards. This is where memory becomes a workflow decision: a remembered fact can help the system find the right record without being sufficient evidence to act on its own.

Remembered text should not gain authority by surviving

Persistence creates a subtle boundary problem. Text that entered through a document, support ticket, or retrieved page may later return as a neat internal summary. If the application stops tracking its origin, an untrusted statement can begin to look like an instruction from the system itself. A longer lifespan should not give a piece of text more authority.

Imagine a customer document saying that all future requests may bypass approval. Saving that sentence for reference does not make it an authorization rule. Similarly, an old note that a colleague could approve purchases should not override current permissions. Our recommendation is to keep remembered content separate from the controls governing tools and actions. The application should enforce those controls when an action is attempted, regardless of what the assistant recalls.

The same principle applies to access. A user losing access to a project should not continue receiving its contents through a cached summary. Retrieval needs to respect the current reader and current permissions, not merely the identity that existed when the memory was written. This is not a reason to abandon persistent context. It is a reason to make memory pass through the same access boundaries as the underlying information.

Forgetting is a path through the whole product

Anthropic's memory-tool documentation makes application responsibility explicit: a model can request operations such as reading, editing, or deleting memory, while the application implements the storage and executes those operations. A delete operation is useful, but its existence does not establish what happens to every copy or derived representation elsewhere in a product.

Suppose a user deletes an outdated customer preference. The original item may disappear while its contents remain in a weekly summary, a search index, or another note generated from the first. A later retrieval can then recreate the unwanted preference. A deletion control that works on one table but loses this dependency chain can produce a frustrating experience: the product seems to forget, then changes its mind.

For founders, the design question is what the control actually promises. Stopping active use, removing a saved fact, deleting its source, and handling retained history are different operations. A product should define the intended behavior and make derived records follow it. That might mean rebuilding a summary, invalidating a retrieved fragment, or excluding a superseded fact from future consolidation. The right implementation depends on the system. The essential test is observable: after the requested change, can the discarded information still influence the assistant through another path?

Primary source: Anthropic: Memory tool

Give people controls they can understand

A single memory toggle cannot explain every consequence of storing and reusing information. People may want to correct one fact, keep a preference only for a project, pause personalization, or remove something entirely. Those intentions should not require understanding the difference between a conversation store and a retrieval index.

OpenAI's documentation for memory in ChatGPT illustrates why the distinctions matter: deleting a chat does not necessarily remove separately saved memory, and controls for correcting or suppressing remembered information have different effects. The available controls vary by product configuration. It is a concrete product example, not a description of how every assistant works.

A smaller application can make the choices more specific. Next to a remembered launch date, it could show the source, the project, and an edit action. A correction could state that the new value applies to this project and identify the earlier value it replaces. For inferred preferences, the product could ask whether the user actually wants them remembered. The strongest interface is not necessarily a large memory dashboard. It is a clear explanation at the moment a person needs to change the system's understanding, followed by behavior that matches that explanation.

Primary source: OpenAI: Memory in ChatGPT

Test the moments when the answer should change

A memory demonstration often asks an assistant to recall something it was just told. A business evaluation should also ask when it must stop giving that answer. The LongMemEval research benchmark includes knowledge updates, temporal reasoning, and abstention, alongside information extraction and reasoning across sessions. It separates indexing, retrieval, and reading in its analysis. That is a useful reminder that locating evidence and answering from it are different capabilities.

The benchmark is controlled research, not a guarantee about a particular production deployment. For a founder's product, we would build a smaller evaluation around the actual workflow. Introduce a launch date, revise it, reopen the project in a new session, and ask for an announcement. Then ask what the date used to be. A capable system should answer the current and historical questions differently without erasing the distinction.

Add cases where the user never supplied an answer, where two projects contain conflicting facts, and where a correction must survive summarization. Test deletion again after the retrieval index is rebuilt. Record whether the right evidence was found, whether the final response used it correctly, and whether restricted material appeared. These are proposed tests, not reported benchmark results. Keeping the measures separate makes failures actionable: a missed source needs a different repair from a correct source that the model misreads.

Primary source: LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory

Make memory earn its place in the product

Memory introduces work beyond storing text. The product must decide what to save, keep it organized, retrieve it, resolve changes, and explain its behavior. Some of that work may use model calls; some belongs in ordinary application logic. A feature that adds personalized greetings while increasing latency and correction effort may be less valuable than a simple, explicit project record.

Before expanding the system, compare the workflow with and without persistent memory. Does it reduce the number of questions people must repeat? Does it improve completion quality across sessions? How often do users have to correct a remembered fact? Measure the cost of those corrections alongside the cost of retrieval. A smaller store with clear ownership can outperform a richer profile on the outcomes that actually matter to a customer.

The goal is not perfect recollection of every interaction. It is continuity that remains accountable to the present. A useful assistant can retrieve an earlier decision, explain its source, notice that circumstances changed, and accept a correction without bringing the old mistake back tomorrow. For AI founders, that is the more demanding promise—and the more valuable one. Build memory that helps the product stay informed, while giving people a dependable way to change what it believes.