A agent forgets between chats. Hierarchical memory keeps lessons cheap to inject and cheap to fix: sessions, episodes, compiled knowledge, then weights.
Overview
Memory is not one store. It is a stack of units with different cost, life, and use.
L0 is the live session. L1 is an episode. L2 is compiled knowledge. L3 is weights.
The job of the stack is to keep the next session cheap and correct.
The move from a lower layer to a higher layer is promotion. Promotion is extraction plus write. It is not a dump of the chat. It is not a new training run.
This piece answers three questions. Why do you need this stack. Why a stack works better than a flat store. How background tasks clean, repair, and organize memory.
These L-numbers are product ranks. They are not Tencent Chat L-numbers. That difference is in Prior research.
Why this is needed
A coding agent repeats work that a person would not repeat. It greps the same tree. It rereads the README. It misses a constraint that the last session already proved. That costs tokens and wall clock. It also fails when the workspace is wrong: a misleading README can beat a fact the agent already earned, if that fact is not on disk and trusted.
The live context window makes the problem worse. Tool logs, search hits, stack traces, and diffs fill the prompt. The agent then reasons over a pile that mixes current state with old evidence. Truncation throws the pile away. A lossy summary throws the evidence away. Neither is a memory system.
Flat vector memory has a different failure. Each turn becomes a fragment. Recall is nearest-neighbor search over disconnected chunks. The store does not know which chunk is current, which chunk is a decision, and which chunk is a dead end. Multi-session work makes this worse. A preference from week one and a constraint from week four sit in the same bag.
You also cannot send every lesson into weights. Fine-tunes are slow, costly, and hard to undo. They do not transfer cleanly when the base model changes. Most of what an agent needs tomorrow is a small set of durable facts and procedures. Those belong in text first.
So the need is practical:
- Keep lessons across sessions.
- Keep the next prompt small.
- Keep a path back to evidence.
- Change fast units fast, and slow units slowly.
The four layers
Think of a triangle. The base is large and cheap to write. The top is small and expensive to change.
L0 is the session. It is the host chat: user turns, tool calls, errors, and diffs. It is ground truth. Many hosts already store it. A memory product does not need a second archive on day one. The session is the source for promotion. It is also the last place you go when a higher layer is not enough.
L1 is the episode. An episode is one atomic unit of experience: a fact, a constraint, a gotcha, a decision that proved true. It is small enough to retrieve. It is easy to edit or mark stale. In our work this unit is a knowledge card: a markdown file with a title and a body. The name “episode” is the cognitive one. The file is the engineering one.
L2 is compiled knowledge. Many episodes, over time, become a procedure, a wiki, or a graph. A procedural skill is distilled know-how for a repeat workflow. It is not the host plugin that teaches the agent how to call tools. A wiki is linked pages that you compile once and keep current. A graph is claims, flows, and file anchors for navigation. You query L2 when the task needs a handbook, not a single fact.
L3 is weights. LoRA or a fine-tune can absorb patterns that text memory keeps restating. Open this layer when L1 and L2 stop moving the metric. Eval it on the same tasks as the text layers. Compare four arms when you can: no memory, text only, weights only, both.
Why hierarchy works
The window is scarce. You cannot inject a week of sessions into every prompt. You can inject a short trusted block: titles, an index, a persona, a skill name. The agent fetches a body when the title is not enough. Tencent calls this progressive disclosure. The same idea shows up as title-first inject for cards, and as index.md first for a wiki.
Layers have different clocks. A session changes every turn. An episode changes when a task ends. A compiled page changes after many tasks. Weights change after a training job. If you mix these clocks in one store, retrieval cannot know what is stable.
Upper layers hold structure. Lower layers hold evidence. A compiled skill says how to do the work. The episode says what proved true on a given day. The session shows the exact error. Keep the chain. Compression without a drill-down path is amnesia.
Wrong memory must be cheap to fix. You can mark one episode STALE. You can rebuild a wiki page. You cannot surgically unlearn one sentence from a LoRA. That is why L3 comes last.
The split is old. Complementary Learning Systems (McClelland and others) describe a fast store for episodes and a slow store for structure. People replay and reorganize between waking tasks. Agents can do the analogue in background jobs. Hierarchy works because it matches that split: fast write at the bottom, slow compile at the top.
Promotion
Promotion is the write path up the triangle.
From L0 to L1, the agent (or a background job) reads the session and writes episodes. Write the outcome that proved true. Do not write the tour of files. Do not write a plan. Do not write a guess. One episode, one fact.
From L1 to L2, a later job reads many episodes and traces. If the same procedure keeps working, compile a skill. If the same entities keep appearing, compile wiki pages. If the same files keep connecting, compile graph claims.
From L2 to L3, a training job may absorb a stable skill into weights. Do this only when the text form is already useful, and you need the model itself to act as if it already knew.
Promotion can run at two times. In flow: the agent proposes an episode when the fact appears. After the session: a follow-up turn or a background worker promotes what the in-flow path missed. End-only promotion misses mid-task gotchas. In-flow-only promotion misses the view from the end. Use both when you can.
A dedicated promoter can rebuild a notebook from a full episode. The live agent can also promote in a follow-up turn, with no extra model in the product. The second path is simpler. The first path is a later A/B, not a requirement for v0.
Background work: clean, repair, organize
A store that only appends will rot. Background tasks are how the stack stays small enough to trust.
Clean. Merge near-duplicates. Drop units that no retrieval uses. Cap how many items enter the prompt (count and characters). Offload verbose tool logs from the live window. Keep an index in context, not the raw log.
Repair. Code changes. Old facts become false. Mark STALE and demote. Let a run confirm or flag an episode so retrieval can bias by trust. Lint a wiki for contradiction, orphans, and gaps. When a failure and a success exist for the same task, a contrastive patch is sharper than a success summary alone.
Organize. Rebuild a notebook instead of appending forever. Keep an index and a chronological log. File skill pages so a bank does not grow dead twins. Group claims with file anchors so planning does not start from grep.
Do this work when the user is not waiting. Letta’s sleep-time compute is the named form: a second agent rewrites memory while the first agent is idle. You can use a slower, stronger model there. The live path stays retrieve-and-act.
Karpathy’s LLM wiki makes the same split in three verbs: ingest, query, lint. Ingest compiles a source into pages. Query reads the wiki. Lint is the health check. Agents do not get bored when they update fifteen pages. People abandon wikis because that bookkeeping is the part they skip.
Prior research
The layer idea did not start with coding agents.
Cognitive science. Endel Tulving split episodic memory (a dated event) from semantic memory (a stable fact). Complementary Learning Systems treat the hippocampus as a fast episode store and the neocortex as a slow structure store. Replay during rest is the biological background job. Agent stacks copy the shape: write fast at L0/L1, compile slowly at L2, change weights last.
MemGPT and Letta. Packer et al. treated the context window as main memory and external stores as disk. The agent pages with tool calls. Letta names three tiers: core (always in the prompt), recall (searchable chat log), archival (external facts). Sleep-time compute (Lin et al., 2025) moves rewrite and precompute off the live path. That is the maintain story: do clean and repair while idle.
TencentDB Agent Memory. Tencent’s open-source hub uses layering plus symbols. Short-term: offload tool logs to files, keep a Mermaid canvas in context, drill down by node_id. Long-term Chat pyramid: L0 conversation, L1 atom, L2 scenario, L3 persona. Retrieval starts at the top and goes down for detail. Hybrid BM25 plus vectors plus RRF is their recall default. They also keep Skill, Wiki, and CodeGraph as hub assets beside that Chat pyramid. Matching digits are not matching units. Their Chat L2 is a scene block. Their Chat L3 is persona.md. Our L2 is compiled skill / wiki / graph. Our L3 is weights. Their skill-generation path (trace to scenario to SOP) is closer to our promotion into compiled skills.
Tencent’s own OpenClaw numbers (their eval, long sessions): WideSearch pass 33% to 50% with a large token drop; SWE-bench 58.4% to 64.2%; PersonaMem 48% to 76%. Treat those as their report, not as a port.
Compiled wikis. Karpathy’s LLM-wiki gist is the compile-once pattern. Raw sources stay immutable. The model writes the wiki. index.md is the catalog. Lint finds stale claims and missing links. Query reads the wiki instead of re-deriving from raw files on every question.
Atomic units and skills. Continual Learning Bench knowledge cards treat small trusted markdown units as the memory the agent must obey. Mem0 extracts, updates, or adds facts instead of appending forever. Trace2Skill, CODESKILL, and SKILL-KD distill traces into portable skills, then maintain the bank: add, modify, delete, skip. Contrastive pairs (failure plus success) yield sharper patches than success-only summaries.
Weights last. Memco-style studies that keep weights frozen and move quality in shared text memory are the caution for L3. If text memory already moves the metric, a LoRA is an extra cost and a lock-in to one model. Skills in files can move to a new model. Weights often cannot.
Conclusion
Hierarchical memory is a way to spend tokens and trust.
You need it because agents forget, because context fills with logs, and because a flat pile cannot tell a decision from a dead end.
It works because each layer has a job: sessions for evidence, episodes for fast lessons, compiled pages for structure, weights for habits that text keeps repeating.
Promotion is the write path up. Background tasks are the health path: clean duplicates, repair stale claims, organize indexes and skills. Do that work when the user is not in the loop.
Build in that order. Make L1 episodes small, trusted, and cheap to inject. Compile L2 when episodes repeat. Touch L3 when the cheaper layers stop moving the score. Keep a path from a compiled claim back to the session that earned it. If you cannot walk that path, you do not have memory. You have a summary you cannot check.
Sources
- TencentCloud/TencentDB-Agent-Memory
- Packer et al., MemGPT: Towards LLMs as Operating Systems, 2023
- Lin et al., Sleep-time Compute, Letta and UC Berkeley, 2025. See also Continual Learning in Token Space
- Karpathy, LLM Wiki
- McClelland, McNaughton, and O’Reilly, Complementary Learning Systems
- Tulving, episodic vs semantic memory
- Asawa et al., Continual Learning Bench knowledge cards
- Trace2Skill; CODESKILL; SKILL-KD (contrastive skill distillation)