How AI Agents Maintain State and Memory Across Long-Running Sessions

Most "agent memory" demos are a parlour trick. The agent remembers your name, your preferences, maybe your coffee order — and then, three days into a support ticket, cheerfully re-runs the diagnostics it already ran and orders the part it already ordered. Remembering a fact is not the same as resuming a task, and the industry keeps confusing the two.

"Memory" is three different systems wearing one trench coat

Session state (what's done, what's pending), working context (what the model can see right now), and long-term memory (facts and episodes stored outside the model). An agent can be excellent at one and absent at another. The failure mode that matters most — losing the ticket, not the trivia — is the one almost nobody demos.

Compaction is a slow leak, not a solution

When the history approaches the context window, the standard answer is to summarise the old turns and keep going. Anthropic's compaction does this, LlamaIndex does this, everyone does this. But a summary keeps the gist, not the detail — and the detail that gets dropped is exactly the decisive fact: what was decided, what's still open, what the next step is. Bigger windows don't fix it either; Anthropic's run from 200K to 1M tokens just delays the same leak.

The checkpoint is the part everyone skips

LangGraph's checkpointer is the clearest worked example of the bit that actually makes multi-day work survive: a snapshot of structured task state — progress, variables, tool results — keyed by a stable thread ID, so a later session resumes without repeating completed work. But note the trap: reusing a thread ID alone never resumes an unfinished run. What starts new work, resumes unfinished work, or replays history is the input and checkpoint ID you pass. Get that wrong and your "resumable" agent is just a chat log with ambitions.

Don't trust the memory benchmarks either

Mem0 claims 92.5% on LoCoMo, Zep claims 94.7% — and an independent audit found about 6.4% of LoCoMo's answer keys are score-corrupting errors, while the judge commonly used to score it accepts roughly 63% of intentionally wrong but topically adjacent answers. A lenient judge conceals recall failures and inflates the score. Vendor numbers on a noisy benchmark are marketing, not rankings.

The honest constraint nobody advertises

No vendor offers a general persistent-memory API for developers. ChatGPT memory is a product feature; Anthropic offers compaction, not cross-session memory. If you want durable memory today, you build it from frameworks like LangGraph and LlamaIndex or managed services like Mem0, Zep and Letta — and you test it the way it will actually be used, multi-session, judged on accuracy alongside latency and token cost.

Start simple. Add persistent memory only when a session boundary actually hurts — when users repeat themselves, when tasks restart from scratch, when the same question gets answered twice. And keep the checkpoint store on a separate path from the memory store: a bad summary costs you a conversation, but a lost checkpoint costs you the task.

Go deeper

Read the full explainer — including the LangGraph checkpoint code, the MemGPT retrieval numbers, and the five-part reference architecture — on Agentic Affairs: How AI Agents Maintain State and Memory Across Long-Running Sessions

Originally published on Agentic Affairs (https://agentic-affairs.com)