Title: Contextual Agentic Memory is a Memo, Not True Memory ArXiv: 2604.27707 Authors: Binyan Xu, ,, Xilin Dai, and Kehuan Zhang, The Chinese University of Hong Kong, Hong Kong, China, Zhejiang University, Hangzhou, China Sections: 28 Estimated tokens: 21.8k ## Contents - 1 Introduction - 2 The Case: Agentic Memory Is a Memo, Not True Memory - 3 Evidence: Four Structural Limitations - 3.1 Definitional: Exemplar-Based Lookup Cannot Extrapolate - 3.2 Structural: The Generalization Gap - Theorem 1 (Compositional Sample Complexity Separation) . - Proof sketch. - Remark 2 (Non-circularity and context-window independence) . - 3.3 Dynamic: The Frozen Novice Problem - 3.4 Security: Persistent Compromise - 4 Alternative Views - 5 Call to Action - 5.1 For System Builders: Build the Consolidation Channel - 5.2 For Benchmark Designers: Measure Learning, Not Recall - 5.3 For the Continual Learning Community: The Agentic Setting Is Your Deployment Target - 6 Related Work - 7 Conclusion - Acknowledgments and Disclosure of Funding - References - Appendix A Proof of Theorem 1 (Performance Ceiling Bound) - Appendix B Proof of Theorem 1 (Compositional Sample Complexity Separation) - Appendix C Proof of Proposition 1 (Information-Theoretic Bound on ICL Accuracy) - Appendix D Comparison of Continual Learning Methods - Appendix E Constructive Example and Composition-Error Corollary - E.1 Modular arithmetic construction - E.2 Composition-error corollary - Assumption 3 (Domain specificity) . - Proposition 4 (Composition-error formulation) . ## Abstract Abstract Current agentic memory systems (vector stores, retrieval-augmented generation, scratchpads, and context-window management) do not implement memory: they implement lookup . We argue that treating lookup as memory is a category error with provable consequences for agent capability, long-term learning, and security. Retrieval generalizes by similarity to stored cases; weight-based memory generalizes by applying abstract rules to inputs never seen before. Conflating the two produces agents that accumulate notes indefinitely without developing expertise, face a provable generalization ceiling on compositionally novel tasks that no increase in context size or retrieval quality can overcome, and are structurally vulnerable to persistent memory poisoning as injected content propagates across all future sessions. Drawing on Complementary Learning Systems theory from neuroscience, we show that biological intelligence solved this problem by pairing fast hippocampal exemplar storage with slow neocortical weight consolidation, and that current AI agents implement only the first half. We formalize these limitations, address four alternative views, and close with a co-existence proposal and a call to action for system builders, benchmark designers, and the memory community. ## 1 Introduction Modern LLM agents remember by writing. MemGPT pages information in and out of context like an OS (Packer et al., 2024); Generative Agents record every observation in a memory stream (Park et al., 2023); Reflexion stores verbal self-critiques in an episodic buffer (Shinn et al., 2023); Voyager accumulates skills as code in a vector database (Wang et al., 2023). Every major agentic framework treats retrieval as the default persistence mechanism (Hu et al., 2026), and the implicit promise is that richer external stores will eventually produce agents with richer inner lives. The pivot to retrieval was not irrational: external stores are reversible, auditable, and safe to deploy—genuinely valuable properties under production constraints. Our argument is not that retrieval was the wrong choice to ship; it was the right choice to ship. But the field has since conflated “best engineering compromise available” with “sufficient substitute for learning,” and this conflation is now a category error with provable consequences. Retrieval generalizes by similarity to stored cases; weight-based memory generalizes by applying abstract rules to inputs the agent has never encountered. Agents built exclusively on retrieval accumulate notes without ever developing expertise, face a provable generalization ceiling on compositionally novel tasks that no increase in context window size can overcome, and structurally convert transient prompt injections into persistent compromise. A Reflexion agent that accumulates thousands of verbal self-critiques is still running the same frozen model at every session; its filing cabinet grows while its capacity does not (Shinn et al., 2023). We argue that this conflation is a category error: treating a memo as a mind. Cognitive science distinguishes *exemplar-based* cognition (generalizing by similarity to stored cases) from *rule-based* cognition (applying abstract principles extracted from, but no longer dependent on, those cases) (Nosofsky et al., 1994; Ashby and Maddox, 2011). The transition between the two is, in the brain, the consolidation of hippocampal episodes into neocortical weights during sleep (McClelland et al., 1995; O’Reilly et al., 2014). Chi et al. (1981) showed that physics novices categorize problems by surface features (“inclined plane problems”) while experts categorize by deep structural principles (“conservation of energy problems”)—a reorganization that is not a change in what they have *stored*, but in how knowledge is *represented*. Current AI agents implement only the hippocampal half; no consolidation path to the neocortical half exists in any deployed system. The stakes are not merely philosophical. Developers who deploy long-running agents under the assumption that accumulated retrieval constitutes growing expertise will find that capability stagnates while attack surface grows. Benchmark designers who evaluate agents on seen task distributions will miss the generalization gap that opens on genuinely novel inputs. The continual learning community, whose methods could close this gap, has largely ceded the agentic setting to retrieval-based approaches (Ke and Liu, 2023). None of these communities can course-correct without first recognizing that retrieval and memory are not interchangeable. We advance four claims. First (Definitional), agentic memory is exemplar-based lookup: it cannot extrapolate to compositionally novel situations that no stored case covers. Second (Structural), we prove a Generalization Gap theorem showing retrieval-based memory has a provably lower ceiling than weight-based memory on novel tasks, independently of context window size or retrieval quality. Third (Dynamic), agents operating exclusively via $C$-engineering cannot develop expertise; each session begins from the same frozen weights. Fourth (Security), agentic memory structurally converts transient prompt injection into persistent compromise. We close with a co-existence architecture and a call to action for system builders, benchmark designers, and the continual learning community. We use *exemplar-based*, *lookup*, and *memo* interchangeably for retrieval-based systems, and *rule-based*, *function*, and *true memory* for weight-based systems; Table [1](#S2.T1) formalizes the distinction. ## 2 The Case: Agentic Memory Is a Memo, Not True Memory **Table 1: Memory taxonomy for LLM agents. All current agentic systems occupy the “Episodic” row. The “Experiential” row (weight-based encoding of the agent’s lived experience) is the gap.** | Type | Substrate | Persists | Updated by | Generalizes | | --- | --- | --- | --- | --- | | Working | Context window | Session only | Token generation | Limited by $L$ | | Episodic | External store | Cross-session | Read/write ops | Exemplar-based | | Semantic | Model weights | Permanent | Pre-training | Rule-based | | Experiential | Model weights | Permanent | Fine-tuning/CL | Rule-based | Table [1](#S2.T1) maps the landscape. The “Experiential” row is systemically absent from all deployed systems (Hu et al., 2026). This absence is not accidental: it reflects a field-wide assumption that agentic memory (the “Episodic” row) is a sufficient substitute. We argue it is not. A unifying lens: two structurally distinct paths. As an analytical lens, every technique that changes what an LLM agent outputs can be characterized as belonging to one of two structurally distinct categories (acknowledging that some mechanisms, such as persistent test-time adapters, sit at the boundary—we return to this in Alternative 5): (1) Change $\theta$: modify the model’s weights via pre-training, fine-tuning, reinforcement learning, or any other gradient-based update, changing the prior distribution $P(X\mid\theta)$; (2) Change $C$: inject content into the context window via prompting, RAG, MCP tool calls, skill files, scratchpads, or any other form of context engineering, conditioning generation on $P(X\mid\theta,C)$. These two paths are *structurally different*, not merely different in implementation. Changing $\theta$ compresses knowledge into the model’s weight space, whose capacity scales with the number of parameters. Changing $C$ compresses knowledge into text, whose capacity is bounded by the context window length $L$. The critical asymmetry is not compression ratio but *generativity*: $\theta$-compression is generative (the model can recombine weight-encoded rules to handle inputs it has never seen); $C$-compression is retrieval-based (the model can only use what is explicitly present in context). *All current deployed agentic memory is $C$-engineering.* Every technique in Table [1](#S2.T1)’s “Episodic” row (MemGPT, RAG, Reflexion, Voyager) changes $C$, not $\theta$. The weights are identical before and after the agent’s experience. The Experience Compression Spectrum. A recent line of work (Zhang et al., 2026b) formalizes memory, skills, and rules as lying on a single *experience compression spectrum*, differing only in their compression ratio: raw traces (low compression, high fidelity) $\to$ natural-language skills (medium compression, actionable) $\to$ parameterized rules (high compression, generalizable). This spectrum supports our position: the field currently implements *all three* (memory, skills, rules) as context-based lookup, confusing points on the spectrum with each other. True skill-based and rule-based cognition require the *higher-compression* endpoint of the spectrum (weight-based encoding), not merely storing low-compression traces or medium-compression skills as retrievable text. The spectrum framing makes the normative claim precise: *the appropriate substrate depends on the compression level* — episodic traces belong in an external store (fast, temporary); skills can live in context or weights (bridge); rules *must* be in weights (slow, generalizable). Current systems implement the entire spectrum as $C$-engineering, which is the root error. What agentic memory actually does. When we say an agent “remembers” using MemGPT or RAG, we mean: (1) during a past session, something was written to an external store; (2) during the current session, a query is issued and similar entries are returned; (3) the returned entries are injected into context. This is a lookup. The model weights are identical before and after the experience. The agent recorded the experience and retrieved it; it did not learn from it. The diary contrast. A person who records a lesson in a diary can retrieve it only if they consult the diary and the situation is similar enough to trigger recall. A person who has internalized the lesson has it available everywhere. Agentic memory is the former; weight-based learning is the latter. Reflexion calls its approach “verbal reinforcement learning,” yet the model’s weights remain unchanged; Voyager stores skills as code yet novel composition still falls back on the frozen base model; Generative Agents write synthesized insights to the memory stream yet the reflecting model is the same frozen model every session. The RAG inversion. Lewis et al. (2021) introduced RAG explicitly framing it as *augmenting* parametric memory with a retrieval index. The field has since progressively replaced parametric updates with expanded non-parametric stores, inverting the original vision. The tacit assumption (more retrieval equals more intelligence) is what the next section formally refutes. The existence proof. ROME localized factual associations to specific MLP layers (Meng et al., 2022); MEMIT edited thousands of facts via weight modification (Meng et al., 2023); ParamMem showed empirically that encoding agent reflections into weights outperforms storing them externally (Yao et al., 2026). Weight-based memory is not speculative; it is already available. The question is what is lost when knowledge lives only in an external store. Cognitive science grounding. Complementary Learning theory (McClelland et al., 1995; O’Reilly et al., 2014) identifies the neural substrate of this distinction: the hippocampus provides fast episodic storage; the neocortex encodes slow, distributed, rule-based representations consolidated during sleep (Brodt et al., 2023). Chi et al. (1981) showed this reorganization is not a change in what novices have *stored* but in how knowledge is *represented*. The AI agent community has invested almost exclusively in the hippocampal side (Hu et al., 2026); the neocortical half remains unaddressed. ## 3 Evidence: Four Structural Limitations ### 3.1 Definitional: Exemplar-Based Lookup Cannot Extrapolate The distinction between exemplar-based and rule-based cognition is not merely terminological. A lookup table maps seen inputs to stored outputs. A function maps *any* input to a principled output, including inputs it has never encountered, by virtue of the abstract structure it has internalized. Human experts are functions; novices with textbooks are lookup tables (Chi et al., 1981). When a chess grandmaster encounters a novel position, they do not retrieve the most similar position from memory and copy its move; they reason from deeply internalized principles that their experience has encoded. When a language agent encounters a novel question for which no retrieved document provides a direct answer, its agentic memory offers nothing. Retrieval generalizes by similarity to stored cases. Rule-based cognition generalizes by applying abstract principles that were extracted from, but no longer dependent on, stored cases. Current agentic memory systems implement exemplar-based cognition in perpetuity; they provide no mechanism for the transition to rule-based cognition. Empirical confirmation follows in Section [3.2](#S3.SS2): Yao et al. (2026) show that parametric storage outperforms retrieval-based storage, with the performance gap growing precisely on novel compositional tasks. ### 3.2 Structural: The Generalization Gap The central theoretical contribution of this paper is a formal account of why $C$-engineering cannot replicate $\theta$-learning for compositionally novel inputs, regardless of how large or sophisticated the retrieval system becomes. A sample-complexity lens. We analyze both memory paradigms through *compositional sample complexity*: given the same budget of domain experience, how well does each system generalize to *unseen combinations* of known concepts? This framing avoids the circularity of defining “what retrieval cannot do” and then proving it cannot; instead, it establishes a *quantitative efficiency gap* under transparent, empirically testable assumptions. Let $\mathcal{F}=\{f_{1},\ldots,f_{k}\}$ be the base concepts in a target domain, and let $\oplus:\mathcal{F}\times\mathcal{F}\to\mathcal{Y}$ be the domain-specific *composition operator* that maps concept pairs to correct task outputs. The agent observes a training set $D$ of $n$ labeled compositional examples $(f_{i},f_{j},\oplus(f_{i},f_{j}))$; the same data is available to both the retrieval store and the fine-tuning pipeline, ensuring a symmetric comparison. Let $M_{\mathrm{P}}$ denote *parametric memory* (fine-tuned weights $\theta^{*}$) and $M_{\mathrm{R}}$ denote *retrieval-based memory* (a store $D$ queried by a frozen model $f_{\theta_{\mathrm{frozen}}}$ with top-$K$ retrieval). A concept pair $(f_{i},f_{j})$ is *compositionally novel* w.r.t. experience $D$ if no entry in $D$ records $\oplus(f_{i},f_{j})$; let $S\subseteq\binom{\mathcal{F}}{2}$ be the pairs covered by $D$ and $N=\binom{k}{2}$ the total. The intuition is immediate: a student who has studied kinematics and thermodynamics separately faces such a novel question when asked to combine both—retrieval returns prior instances in isolation, so the composition rule must be stored exhaustively or learned parametrically. Assumption 1 (Bounded in-context composition). We assume that the frozen model $f_{\theta_{\mathrm{frozen}}}$, given any $K$ demonstrations of $\oplus$ from $D$, achieves accuracy $\alpha(K)\leq\bar{\alpha}<1$ on held-out pairs—as holds whenever $\oplus$ was absent or underrepresented at pretraining (post-cutoff clinical protocols, enterprise legal conventions, specialized engineering standards). When $\oplus$ is broadly general, $\bar{\alpha}\to 1$ and the separation vanishes; the assumption binds precisely in the domain-specific deployments where persistent agents are most valuable. This assumption is not merely plausible; Appendix [C](#A3) shows via Fano’s inequality (Cover and Thomas, 2006) that $\bar{\alpha}<1$ is a theorem for any operator class with $\log|\mathcal{H}|>K\log|\mathcal{Y}|$, and reduces to an empirical claim only when $K$ examples already suffice to identify $\oplus$—the regime where no learning of any kind is needed. We measure generalization via the *Compositional Generalization Capacity* $\operatorname{CGC}(M,D)$: accuracy of memory system $M$ over all concept pairs drawn uniformly from $\binom{\mathcal{F}}{2}$. ###### Theorem 1 (Compositional Sample Complexity Separation) . Under Assumption [3.2](#S3.SS2), for any target generalization level $1-\delta$ with $\delta<1-\bar{\alpha}$: - 1. Retrieval lower bound. Any retrieval-based memory $M_{\mathrm{R}}$ with frozen model $f_{\theta_{\mathrm{frozen}}}$ requires $n_{R}\;\geq\;\frac{1-\delta-\bar{\alpha}}{1-\bar{\alpha}}\;\binom{k}{2}\;=\;\Omega(k^{2})$ stored compositional examples to achieve $\operatorname{CGC}(M_{\mathrm{R}},D)\geq 1-\delta$. - 2. Parametric upper bound. If $\oplus$ belongs to a hypothesis class $\mathcal{H}$ with VC dimension $d$, fine-tune on $n_{P}\;=\;O\!\left(\frac{d+\log(1/\delta)}{\delta}\right)$ compositional examples suffices for $\operatorname{CGC}(M_{\mathrm{P}},D)\geq 1-\delta$. - 3. Separation. The sample complexity ratio satisfies $n_{R}/n_{P}=\Omega\bigl(k^{2}/d\bigr)$. For structured operators with $d=O(k)$, the ratio is $\Omega(k)$; for operators with $d=O(1)$ (e.g. parameterized group operations), it is $\Omega(k^{2})$. Consequently: $$ $\sup_{M_{\mathrm{R}}}\;\operatorname{CGC}(M_{\mathrm{R}},\,D)\;<\;\sup_{M_{\mathrm{P}}}\;\operatorname{CGC}(M_{\mathrm{P}},\,D)$ $$ whenever the data budget $n$ falls in the gap $n_{P}\leq nk_{0}(d,\delta,\bar{\alpha})$. ###### Proof sketch. For a stored pair $(f_{i},f_{j})\in S$, retrieval answers correctly. For a novel pair, the frozen model processes $K$ retrieved examples but by Assumption [3.2](#S3.SS2) achieves accuracy at most $\bar{\alpha}$, giving $\operatorname{CGC}(M_{\mathrm{R}},D)\leq n/N+(1-n/N)\bar{\alpha}$; solving for $\operatorname{CGC}\geq 1-\delta$ yields the $\Omega(k^{2})$ bound on $n_{R}$. Retrieval faces a *coverage* problem (each stored example covers one point in the $N$-sized space); parametric learning faces a *learning* problem (identify $\oplus$ from $d$-dimensional examples); the asymptotic gap between $k^{2}$ and $d$ cannot be closed by retrieval engineering. Full proof and a constructive modular-arithmetic example are in Appendix [B](#A2). ∎ ###### Remark 2 (Non-circularity and context-window independence) . Unlike formulations that define “what retrieval cannot do” and prove it cannot, Theorem [1](#Thmtheorem1) derives a quantitative sample-complexity ratio from Assumption [3.2](#S3.SS2); Appendix [C](#A3) shows $\bar{\alpha}<1$ is itself a theorem for any operator class with $\log|\mathcal{H}|>K\log|\mathcal{Y}|$, making the argument self-contained. Increasing $K$ may raise $\bar{\alpha}$ marginally, but the $\Omega(k^{2})$ coverage requirement remains as long as $\bar{\alpha}<1$; the separation is therefore independent of context window size. The same bound applies when the retrieval policy is learned (as in MemRL-style systems (Zhang et al., 2026a; Zhou et al., 2025)): optimizing *which* exemplars to retrieve improves $\bar{\alpha}$ without removing the coverage lower bound; only encoding the composition rule $\oplus$ into weights resolves the gap entirely. Assumption [3.2](#S3.SS2) does not require $\bar{\alpha}$ to be constant in $K$; it requires only $\bar{\alpha}<1$, which Appendix [C](#A3) guarantees for any operator class with $\log|\mathcal{H}|>K\log|\mathcal{Y}|$. Mechanistic support. The sample-complexity separation is independently supported by mechanistic interpretability. Meng et al. (2022) show factual associations are stored in specific MLP weight layers, and Geva et al. (2020) show FFN layers act as key-value memories—both implying weight-based storage achieves compact, generalizable representations that retrieval cannot replicate structurally. Yao et al. (2024) further localize knowledge to *fact memory units* in FFN neurons, and show that standard SFT *does not* modify these units: SFT updates only attention routers (how to access existing knowledge), leaving knowledge itself untouched. Ye et al. (2025) quantify this: 90% of SFT parameter updates contribute nothing to knowledge enhancement. This reinforces the call for targeted consolidation (Section [5](#S5)) that updates knowledge circuits directly. Empirical evidence. The theoretical gap is corroborated by converging empirical results. Ovadia et al. (2024) found that RAG excels at rare-entity recall but cannot improve compositional reasoning beyond the base model’s capacity, while fine-tuning improves reasoning systematically even without retrieved documents; Yang et al. (2026) confirm that fine-tuning achieves highest overall accuracy on multi-hop queries requiring novel compositional combinations. Most directly, Yao et al. (2026) compared agents that store reflective experience externally versus parametrically: parametric storage outperforms external storage, with the gap *growing* on tasks requiring transfer to held-out question types (exactly the regime predicted by Theorem [1](#Thmtheorem1)). The compositional generalization benchmarks SCAN (Lake and Baroni, 2017) and COGS (Kim and Linzen, 2020) corroborate this: weight-based models consistently outperform retrieval-only approaches on systematic compositional splits in which novel combinations must be inferred rather than looked up. A complementary capacity bound. Theorem [1](#Thmtheorem1) concerns compositional novelty. A separate capacity constraint (Appendix [A](#A1)) shows that for any task requiring integration of $m>K$ mutually dependent facts, $\sup_{M_{\mathrm{R}}}\operatorname{Acc}(M_{\mathrm{R}},\mathcal{T}_{m})<\sup_{M_{\mathrm{P}}}\operatorname{Acc}(M_{\mathrm{P}},\mathcal{T}_{m})$; “Lost in the Middle” (Liu et al., 2023) and Paulsen (2026) provide empirical corroboration. Theorem [1](#Thmtheorem1) is the deeper constraint: it persists even if the capacity constraint is removed. ###### Theorem 1 (Compositional Sample Complexity Separation) . ###### Proof sketch. ###### Remark 2 (Non-circularity and context-window independence) . ### 3.3 Dynamic: The Frozen Novice Problem Theorem [1](#Thmtheorem1) describes a static property of retrieval-based memory. The frozen novice problem describes a *dynamic* consequence: agents that operate exclusively via $C$-engineering (changing context, never changing $\theta$) cannot develop expertise over time. Every session begins with the same frozen weights; the agent is permanently doing .predict(C), never .train(). No matter how many experiences are logged to the external store, the weights encoding the agent’s composition rules remain those of the original pre-trained model. The most robust finding in cognitive science is that expertise does not emerge from accumulating more examples, but from a structural reorganization of knowledge (Chi et al., 1981; Bransford et al., 2000). Novices represent domains by surface features (“this looks like a problem with an inclined plane”); experts represent domains by deep structural principles (“this is a conservation-of-energy problem”). This reorganization requires weight changes in the brain: the formation of generalized, distributed representations in the neocortex through repeated consolidation of hippocampal traces (McClelland et al., 1995; Brodt et al., 2023). An agent that accumulates experience only through agentic memory cannot make this transition. Each session it is the same model with a larger database. Its capacity to apply novel combinations of its knowledge is bounded by the base model’s pretrained composition capacity, not by anything it has learned from experience. The MemGPT team acknowledge that “simply appending raw experience is a poor approximation of learning” and propose “sleep-time compute” as a remedy (Packer, 2025)—but their consolidation rewrites context tokens, not weights. Compressing text in an external store produces better-formatted notes; the agent is still a well-organized novice. The AI analog of CLS consolidation is offline fine-tuning on distilled agent experience (Brodt et al., 2023): the moment the agent’s experience changes what the model *is*, not merely what it *has written down*. In the language of Theorem [1](#Thmtheorem1), each new episode increases $n$ but does not change $\bar{\alpha}$; the coverage requirement $n_{R}=\Omega(k^{2})$ remains, so the agent’s generalization capacity does not improve with experience. ### 3.4 Security: Persistent Compromise The all-inputs-are-evil principle. Classic security engineering holds that well-designed systems treat all external inputs as potentially adversarial (Saltzer and Schroeder, 1975). For LLM agents this is particularly acute: Greshake et al. (2023) showed that when agents retrieve web pages, emails, or documents, embedded instructions can silently hijack behavior (the line between data and instructions is invisible to the model). This prompt injection threat is already well-documented. What has not been sufficiently appreciated is how *agentic memory makes it catastrophically worse*. The evil^2 compounding effect. Without persistent memory an injection is *transient*: one session, then clean. With agentic memory the injected content is written to the store and retrieved in every subsequent session, converting a one-time behavioral hijack ($\text{evil}^{1}$) into a persistent one ($\text{evil}^{2}$). The empirical evidence is striking. MINJA (Dong et al., 2026) achieved a 98.2% injection success rate with injected instructions persisting across sessions at minimal utility cost; PoisonedRAG (Zou et al., 2024) shows that five adversarial texts per targeted query achieve 90% attack success against a knowledge base of millions of entries; and InjecAgent (Zhan et al., 2024) benchmarks indirect injection across 30 LLM agents using 17 user-tool categories, finding memory-writing agents systematically more vulnerable than stateless ones. Attack surface asymmetry. Let $p_{0}$ be the per-interaction probability of a successful injection, and $N(t)$ the cumulative interactions involving external content up to time $t$. For a stateless agent (no persistent memory), the compromise probability per session is bounded by $p_{0}$. For an agent with persistent agentic memory, injected content written to the store is retrieved in every subsequent session; once injected, $p_{0}$ is no longer the per-session risk but the one-time probability of reaching the store, so $P(\text{compromised by }t)=1-(1-p_{0})^{N(t)}\to 1$ as $N(t)\to\infty$. Attack surface asymmetry. Compromising $C$ requires only a single successful injection during normal operation: injected content poisons every future session. Compromising $\theta$ requires training-time access or weight editing (Meng et al., 2022)—capabilities unavailable through normal queries. This asymmetry holds structurally, independent of any assumed $p_{0}$; provenance-based defenses can reduce $p_{0}$ but leave a structurally growing attack surface that audit cannot resolve at scale. Anomalous weight patterns are detectable through activation analysis; anomalous memory entries require semantic audit of a store that grows unboundedly. This security argument reinforces Section [5](#S5): persistent information should migrate to weights, not accumulate in a store whose entries the model cannot distinguish from adversarial content. ## 4 Alternative Views We identify four credible alternatives and explain why each falls short. Alternative 1: Context windows will grow large enough to close the gap. This conflates the *capacity constraint* (Appendix [A](#A1)) with the *compositional generalization constraint* (Theorem [1](#Thmtheorem1)). The capacity constraint is context-sensitive: longer windows hold more facts. But the compositional constraint is context-independent: a novel composition requires the abstract rule that combines concepts, which retrieval cannot provide regardless of window size. In the language of Theorem [1](#Thmtheorem1), larger contexts raise $\bar{\alpha}$ marginally without changing the $\Omega(k^{2})$ coverage requirement; the frozen novice grows a larger filing cabinet, not a sharper mind. Paulsen (2026) show effective context utilization saturates at ${\approx}20$k tokens even for 128k-context models. Alternative 2: In-context learning already implements rule extraction. Large models are remarkable in-context learners, and one might argue that well-chosen exemplars elicit the abstract rule without any weight update. We accept that ICL is powerful, but it activates *parametric rules already encoded during pretraining*; it does not create new ones. When a domain-specific composition pattern is absent from pretraining, ICL has nothing to activate. Akyürek et al. (2023) and Dai et al. (2023) show that under linear self-attention, ICL implicitly implements gradient descent. If so, one might argue ICL already constitutes weight-based learning. Two replies: (1) the implicit update is *ephemeral*—base weights are unchanged at session end, so the same parametric state recurs next session; (2) the equivalence breaks for modern multi-layer Transformers with softmax attention and LayerNorm. The result *supports* our position: the correct fix is to make the gradient-like update persistent—the consolidation pipeline we propose. The argument is strongest where frontier models lack pretraining coverage of the target domain. Alternative 3: Memory, skills, and rules lie on a single compression spectrum, so the distinction is unimportant. Zhang et al. (2026b) unify memory, skills, and rules as points on an experience compression spectrum, differing only in compression ratio. One might argue this undermines our $C$/$\theta$ distinction. We reply: the spectrum framing *supports* our position. If different compression levels require different substrates—low-compression traces in external stores, high-compression rules in weights—then treating all three as context lookup is precisely the error we identify. Voyager (Wang et al., 2023) illustrates this: skills reside in $C$, but the composition rule combining them must come from $\theta$; as compression increases, the substrate must transition from $C$ to $\theta$. Remaining at $C$ for high-compression rules is the category error we identify. Alternative 4: Learned-retrieval systems that optimize what to retrieve can close the generalization gap without weight updates. Recent systems such as MemRL (Zhang et al., 2026a) and Memento (Zhou et al., 2025) learn a retrieval *policy* $\pi_{R}$ rather than relying on static similarity, demonstrating sustained performance improvements without weight updates. We argue it does not, for a structural reason. A learned retrieval policy optimizes *which* exemplars from $D$ to surface; it does not change the fact that the frozen base model processes those exemplars at inference time. For any compositionally novel pair $(f_{i},f_{j})\notin S$, even the optimal policy provides $K$ exemplars, and Assumption [3.2](#S3.SS2) bounds the frozen model’s composition accuracy at $\bar{\alpha}<1$; the $\Omega(k^{2})$ coverage requirement therefore persists. Performance gains are better explained by reduced retrieval noise for *seen* compositions than by generalization to novel ones. Oracle-conditioned episodic retrieval (Lampinen et al., 2025) confirms this: ideal retrieval unlocks abilities *already encoded in pretrained weights*—the $\bar{\alpha}\to 1$ boundary where the separation narrows, consistent with our framework. Learned retrieval is a genuine improvement to the episodic half, but not a substitute for the consolidation channel. ## 5 Call to Action The limitations we document are not inevitable. Agentic memory and parametric learning are complementary, not competing: the right architecture combines fast episodic lookup (retrieval) with a consolidation channel that periodically encodes distilled experience into weights—the AI analog of biological sleep. We address three communities with one clear ask each. ### 5.1 For System Builders: Build the Consolidation Channel The field has spent three years engineering better retrieval. The next step is not better retrieval; it is a second pathway: from the episodic store to model weights. Three design principles follow from our analysis. (1) Agentic memory is episodic lookup—treat it as such. Vector stores and RAG are the right tools for recent context, tool outputs, and reference retrieval. They should not be expected to produce generalization, and system architectures should not be designed as if they will. (2) A consolidation channel must exist. The architecture must include a pathway from the episodic store to parametric memory. The specific mechanism—periodic fine-tuning, knowledge editing, test-time training, self-distillation from traces (Wang et al., 2026)—is a design choice. What matters is that the pathway exists and runs asynchronously, so the agent operates from agentic memory while consolidation proceeds in the background (exactly as biological sleep does not interrupt wakefulness). The required building blocks (LoRA (Hu et al., 2022), SSR (Huang et al., 2024), MEMIT (Meng et al., 2023), TTT layers (Sun et al., 2025), Nested Learning (Behrouz et al., 2026), Skill-SD (Wang et al., 2026)) already exist. The community faces a design choice, not a feasibility barrier. (3) Consolidation must be safe. An obvious objection: a consolidation pipeline that ingests poisoned traces converts a $C$-level attack into a $\theta$-level compromise. This does not undermine the security argument; it sharpens the design requirement. The key insight is that $\theta$-level safety is *auditable in ways that $C$-level safety is not*: weight checkpoints can be versioned and rolled back at bounded cost, whereas identifying and expunging poisoned entries from an unboundedly growing episodic store is intractable. The consolidation pipeline therefore requires trace provenance (so the origin of every distilled experience is auditable), versioned checkpoints (so any bad consolidation can be rolled back), and regression guards (so consolidation is blocked when downstream metrics degrade). These are engineering requirements, not open research problems; they restore the auditability that motivated the original pivot to episodic storage, while extending it to the parametric level. What is genuinely hard. The open questions are not about mechanism but about *policy*: which experiences are worth consolidating (not all traces carry generalizable signal), when to consolidate (too early risks overfitting to noise; too late accumulates stale traces), and how to validate that consolidation produced genuine compositional generalization rather than rote memorization of distilled examples. These questions define the research frontier for system builders. ### 5.2 For Benchmark Designers: Measure Learning, Not Recall Current agentic memory benchmarks (LoCoMo (Wu et al., 2026), LongMem) test how well agents *recall* past events. They do not test whether agents *learn* from past experience. A system that accumulates a better-organized filing cabinet can score near-perfectly on recall benchmarks while exhibiting zero genuine learning—and the field has no way to detect this. The field is optimizing the wrong metric. Benchmark designers have focused on recall quality, context window utilization, and retrieval accuracy. These measure *lookup quality*, not *learning quality*. The result is a benchmark ecosystem that rewards better filing cabinets while saying nothing about whether agents develop expertise. We propose two redirections. Redirection 1: Compositional Generalization over Time (CGT). The single most important metric the field is missing: *does an agent’s ability to handle novel concept combinations improve with experience?* Concretely, operate an agent on a domain for $T$ sessions, exposing it to concepts only in isolation. After $T$ sessions, evaluate on queries requiring combining two or more concepts in ways not seen during operation. A genuinely learning agent shows accuracy strictly increasing with $T$; a pure-retrieval agent shows accuracy flat at baseline. We ask LoCoMo, LongMem, and AgentBench to adopt CGT as a standard evaluation dimension. Redirection 2: Beyond SFT evaluation. The mechanistic evidence in Section [3.2](#S3.SS2) shows that standard SFT modifies attention routers but not fact memory units (Yao et al., 2024; Ye et al., 2025). Yet every current benchmark that evaluates “knowledge update” does so via SFT. This means the field is measuring *access rearrangement* and calling it *learning*. Future benchmarks should evaluate knowledge update via targeted weight editing, circuit-aware fine-tuning, and self-distillation from agent traces—methods that modify the representations where knowledge actually resides. What not to do. Do not benchmark “memory capacity” by context window length. Longer context does not address the compositional generalization gap (Theorem [1](#Thmtheorem1)); it enlarges the filing cabinet. Prioritize generalization over capacity. ### 5.3 For the Continual Learning Community: The Agentic Setting Is Your Deployment Target The field’s pivot from continual learning to agentic memory has left continual learning researchers without a natural deployment context for their methods (Ke and Liu, 2023). We argue that the agentic setting is precisely this context—and that the disconnect is costly for both sides. Agentic systems provide what continual learning methods need: a natural experience stream with reward labels (task success or failure), a clear generalization criterion (compositional novelty), and immediate practical value (deployed agents that should improve over time). Continual learning provides what agentic systems lack: the consolidation machinery (replay, regularization, progressive expansion) that converts episodic experience into durable knowledge without catastrophic forgetting. The research agenda that follows is concrete. *Experience selection*: not all agent traces carry generalizable signal; the CL community’s work on coreset selection and influence functions is directly applicable to deciding what to consolidate. *Consolidation scheduling*: the trade-off between consolidation frequency and stability mirrors the online-vs-batch trade-off in continual learning; existing scheduling theory (Kirkpatrick et al., 2016) applies. *Validation*: how to verify that consolidation produced genuine compositional generalization rather than rote memorization of distilled examples is an open empirical question that the CL community is uniquely positioned to answer. The agentic memory ecosystem is not a competing paradigm; it is the fast-learning half waiting for its slow-learning complement. ## 6 Related Work All major deployed agent memory architectures implement $C$-engineering: MemGPT (Packer et al., 2024), Generative Agents (Park et al., 2023), Reflexion (Shinn et al., 2023), and Voyager (Wang et al., 2023) treat retrieval as the default persistence mechanism; more recent systems (MemoryBank (Zhong et al., 2024), A-MEM (Xu et al., 2025), mem0 (Chhikara et al., 2025)) continue the same paradigm with richer indexing. Surveys (Zhang et al., 2024; Hu et al., 2026; Wu et al., 2026) catalog this space but do not explain why retrieval is structurally bounded. Empirically, Ovadia et al. (2024) and Yang et al. (2026) find fine-tuning improves compositional reasoning while RAG does not; Yao et al. (2026) show the parametric advantage grows on novel compositional transfers—exactly the regime Theorem [1](#Thmtheorem1) predicts—and SCAN (Lake and Baroni, 2017) and COGS (Kim and Linzen, 2020) corroborate weight-based superiority on systematic compositional splits. RETRO and kNN-LM (Borgeaud et al., 2021; Khandelwal et al., 2020) cover static corpora but not post-deployment accumulation; Yu et al. (2026) and MemOS (Li et al., 2025) realize variants of consolidation channels. On security, Greshake et al. (2023), MINJA (Dong et al., 2026), PoisonedRAG (Zou et al., 2024), and InjecAgent (Zhan et al., 2024) document injection attacks against memory-augmented agents; we are the first to formalize persistent compromise as a convergent probability bound (Proposition [3.4](#S3.SS4)). Zhang et al. (2026b) unify memory, skills, and rules as an experience compression spectrum; we sharpen this into a substrate prescription. Boundary cases (Lampinen et al., 2025; Zhang et al., 2026a; Zhou et al., 2025) correspond to the $\bar{\alpha}\to 1$ regime of Assumption [3.2](#S3.SS2) where Theorem [1](#Thmtheorem1)’s separation narrows as predicted; Wang et al. (2026) independently implement the Trace$\to$Skill$\to$FT pipeline we propose. Mechanistic interpretability (Meng et al., 2022, 2023) and continual learning methods (Kirkpatrick et al., 2016; Huang et al., 2024; Hu et al., 2022) supply the consolidation toolbox; this paper provides the theoretical and security motivation. ## 7 Conclusion Agentic memory is a lookup table, not a function: it generalizes by similarity, not by abstract rules. We have shown this entails a provable Generalization Gap on compositionally novel tasks, a Frozen Novice dynamic in which agents accumulate notes but never develop expertise, and a security compounding effect that converts transient injections into persistent compromise. The fix is architectural: pair fast episodic retrieval with an offline consolidation pipeline that encodes distilled experience into weights. Agents are not getting better memories; they are getting better filing systems. Better filing does not make experts; weight-based consolidation does. The position we have staked has immediate methodological consequences. System builders should build the missing consolidation channel—the pathway from agent experience to model weights. Benchmark designers should measure Compositional Generalization over Time: does performance on novel concept combinations improve with experience? The continual learning community should re-engage the agentic setting, which provides the experience stream and success criterion their methods require. The experience compression spectrum (Zhang et al., 2026b) makes the prescription precise: as compression increases, the substrate must transition from $C$ to $\theta$; remaining at $C$ for high-compression rules is the category error we have identified. ## Acknowledgments and Disclosure of Funding Anonymous submission; acknowledgments omitted per double-blind review policy. ## References - E. Akyürek, D. Schuurmans, J. Andreas, T. Ma, and D. Zhou (2023) What learning algorithm is in-context learning? investigations with linear models. In The Eleventh International Conference on Learning Representations, External Links: [Link](https://openreview.net/forum?id=0g0X4H8yN4I) Cited by: [§4](#S4.p4.1). - F. G. Ashby and W. T. Maddox (2011) Human category learning 2.0. Annals of the New York Academy of Sciences 1224 (1), pp. 147–161. External Links: [Document](https://dx.doi.org/https%3A//doi.org/10.1111/j.1749-6632.2010.05874.x), [Link](https://nyaspubs.onlinelibrary.wiley.com/doi/abs/10.1111/j.1749-6632.2010.05874.x), https://nyaspubs.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1749-6632.2010.05874.x Cited by: [§1](#S1.p3.1). - A. Behrouz, M. Razaviyayn, P. Zhong, and V. Mirrokni (2026) Nested learning: the illusion of deep learning architectures. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: [Link](https://openreview.net/forum?id=nbMeRvNb7A) Cited by: [§5.1](#S5.SS1.p3.1). - S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. van den Driessche, J. Lespiau, B. Damoc, A. Clark, D. de Las Casas, A. Guy, J. Menick, R. Ring, T. W. Hennigan, S. Huang, L. Maggiore, C. Jones, A. Cassirer, A. Brock, M. Paganini, G. Irving, O. Vinyals, S. Osindero, K. Simonyan, J. W. Rae, E. Elsen, and L. Sifre (2021) Improving language models by retrieving from trillions of tokens. In International Conference on Machine Learning, External Links: [Link](https://api.semanticscholar.org/CorpusID:244954723) Cited by: [§6](#S6.p1.1). - J. D. Bransford, A. L. Brown, and R. R. Cocking (2000) How people learn: brain, mind, experience, and school. National Academy Press, Washington, DC. Cited by: [§3.3](#S3.SS3.p3.1). - S. Brodt, M. Inostroza, N. Niethard, and J. Born (2023) Sleep-a brain-state serving systems memory consolidation.. Neuron 111 7, pp. 1050–1075. External Links: [Link](https://api.semanticscholar.org/CorpusID:257956951) Cited by: [§2](#S2.p8.1), [§3.3](#S3.SS3.p3.1), [§3.3](#S3.SS3.p5.3). - P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav (2025) Mem0: building production-ready ai agents with scalable long-term memory. External Links: 2504.19413, [Link](https://arxiv.org/abs/2504.19413) Cited by: [§6](#S6.p1.1). - M. T. H. Chi, P. J. Feltovich, and R. E. Glaser (1981) Categorization and representation of physics problems by experts and novices. Cognitive Science 5, pp. 121–152. External Links: [Link](https://api.semanticscholar.org/CorpusID:44436293) Cited by: [§1](#S1.p3.1), [§2](#S2.p8.1), [§3.1](#S3.SS1.p1.1), [§3.3](#S3.SS3.p3.1). - T. M. Cover and J. A. Thomas (2006) Elements of information theory (2. ed.). 2nd edition, Wiley-Interscience. External Links: [Link](https://api.semanticscholar.org/CorpusID:702542) Cited by: [Appendix C](#A3.p5.2), [§3.2](#S3.SS2.p6.4). - D. Dai, Y. Sun, L. Dong, Y. Hao, S. Ma, Z. Sui, and F. Wei (2023) Why can GPT learn in-context? language models secretly perform gradient descent as meta-optimizers. In Findings of the Association for Computational Linguistics: ACL 2023, A. Rogers, J. Boyd-Graber, and N. Okazaki (Eds.), Toronto, Canada, pp. 4005–4019. External Links: [Link](https://aclanthology.org/2023.findings-acl.247/), [Document](https://dx.doi.org/10.18653/v1/2023.findings-acl.247) Cited by: [§4](#S4.p4.1). - S. Dong, S. Xu, P. He, Y. Li, J. Tang, T. Liu, H. Liu, and Z. Xiang (2026) Memory injection attacks on LLM agents via query-only interaction. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: [Link](https://openreview.net/forum?id=QINnsnppv8) Cited by: [§3.4](#S3.SS4.p2.3), [§6](#S6.p2.3). - M. Geva, R. Schuster, J. Berant, and O. Levy (2020) Transformer feed-forward layers are key-value memories. ArXiv abs/2012.14913. External Links: [Link](https://api.semanticscholar.org/CorpusID:229923720) Cited by: [§3.2](#S3.SS2.p8.1). - K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz (2023) Not what you’ve signed up for: compromising real-world llm-integrated applications with indirect prompt injection. Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security. External Links: [Link](https://api.semanticscholar.org/CorpusID:258546941) Cited by: [§3.4](#S3.SS4.p1.1), [§6](#S6.p2.3). - E. J. Hu, yelong shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, External Links: [Link](https://openreview.net/forum?id=nZeVKeeFYf9) Cited by: [Table 2](#A4.T2.4.4.6), [§5.1](#S5.SS1.p3.1), [§6](#S6.p2.3). - Y. Hu, S. Liu, Y. Yue, G. Zhang, B. Liu, F. Zhu, J. Lin, H. Guo, S. Dou, Z. Xi, S. Jin, J. Tan, Y. Yin, J. Liu, Z. Zhang, Z. Sun, Y. Zhu, H. Sun, B. Peng, Z. Cheng, X. Fan, J. Guo, X. Yu, Z. Zhou, Z. Hu, J. Huo, J. Wang, Y. Niu, Y. Wang, Z. Yin, X. Hu, Y. Liao, Q. Li, K. Wang, W. Zhou, Y. Liu, D. Cheng, Q. Zhang, T. Gui, S. Pan, Y. Zhang, P. Torr, Z. Dou, J. Wen, X. Huang, Y. Jiang, and S. Yan (2026) Memory in the age of ai agents. External Links: 2512.13564, [Link](https://arxiv.org/abs/2512.13564) Cited by: [§1](#S1.p1.1), [§2](#S2.p1.1), [§2](#S2.p8.1), [§6](#S6.p1.1). - J. Huang, L. Cui, A. Wang, C. Yang, X. Liao, L. Song, J. Yao, and J. Su (2024) Mitigating catastrophic forgetting in large language models with self-synthesized rehearsal. ArXiv abs/2403.01244. External Links: [Link](https://api.semanticscholar.org/CorpusID:268230393) Cited by: [Table 2](#A4.T2.7.7.5), [§5.1](#S5.SS1.p3.1), [§6](#S6.p2.3). - Z. Ke and B. Liu (2023) Continual learning of natural language processing tasks: a survey. External Links: 2211.12701, [Link](https://arxiv.org/abs/2211.12701) Cited by: [§1](#S1.p4.1), [§5.3](#S5.SS3.p1.1). - U. Khandelwal, O. Levy, D. Jurafsky, L. Zettlemoyer, and M. Lewis (2020) Generalization through memorization: nearest neighbor language models. In International Conference on Learning Representations, External Links: [Link](https://openreview.net/forum?id=HklBjCEKvH) Cited by: [§6](#S6.p1.1). - N. Kim and T. Linzen (2020) COGS: a compositional generalization challenge based on semantic interpretation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), B. Webber, T. Cohn, Y. He, and Y. Liu (Eds.), Online, pp. 9087–9105. External Links: [Link](https://aclanthology.org/2020.emnlp-main.731/), [Document](https://dx.doi.org/10.18653/v1/2020.emnlp-main.731) Cited by: [§3.2](#S3.SS2.p9.1), [§6](#S6.p1.1). - J. Kirkpatrick, R. Pascanu, N. C. Rabinowitz, J. Veness, G. Desjardins, A. A. Rusu, K. Milan, J. Quan, T. Ramalho, A. Grabska-Barwinska, D. Hassabis, C. Clopath, D. Kumaran, and R. Hadsell (2016) Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114, pp. 3521 – 3526. External Links: [Link](https://api.semanticscholar.org/CorpusID:4704285) Cited by: [Table 2](#A4.T2.6.6.5), [§5.3](#S5.SS3.p3.1), [§6](#S6.p2.3). - B. M. Lake and M. Baroni (2017) Generalization without systematicity: on the compositional skills of sequence-to-sequence recurrent networks. In International Conference on Machine Learning, External Links: [Link](https://api.semanticscholar.org/CorpusID:46761158) Cited by: [§3.2](#S3.SS2.p9.1), [§6](#S6.p1.1). - A. K. Lampinen, M. Engelcke, Y. Li, A. Chaudhry, and J. L. McClelland (2025) Latent learning: episodic memory complements parametric learning by enabling flexible reuse of experiences. External Links: 2509.16189, [Link](https://arxiv.org/abs/2509.16189) Cited by: [§4](#S4.p6.7), [§6](#S6.p2.3). - P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela (2021) Retrieval-augmented generation for knowledge-intensive nlp tasks. External Links: 2005.11401, [Link](https://arxiv.org/abs/2005.11401) Cited by: [§2](#S2.p6.1). - Z. Li, S. Song, H. Wang, S. Niu, D. Chen, J. Yang, C. Xi, H. Lai, J. Zhao, Y. Wang, J. Ren, Z. Lin, J. Huo, T. Chen, K. Chen, K. Li, Z. Yin, Q. Yu, B. Tang, H. Yang, Z. J. Xu, and F. Xiong (2025) MemOS: an operating system for memory-augmented generation (mag) in large language models. External Links: 2505.22101, [Link](https://arxiv.org/abs/2505.22101) Cited by: [§6](#S6.p1.1). - N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang (2023) Lost in the middle: how language models use long contexts. Transactions of the Association for Computational Linguistics 12, pp. 157–173. External Links: [Link](https://api.semanticscholar.org/CorpusID:259360665) Cited by: [Appendix A](#A1.p2.8), [§3.2](#S3.SS2.p10.2). - J. L. McClelland, B. L. McNaughton, and R. C. O’Reilly (1995) Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory.. Psychological review 102 3, pp. 419–457. External Links: [Link](https://api.semanticscholar.org/CorpusID:2832081) Cited by: [§1](#S1.p3.1), [§2](#S2.p8.1), [§3.3](#S3.SS3.p3.1). - K. Meng, D. Bau, A. Andonian, and Y. Belinkov (2022) Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems. External Links: [Link](https://api.semanticscholar.org/CorpusID:255825985) Cited by: [Appendix A](#A1.p3.2), [Table 2](#A4.T2.8.8.5), [§2](#S2.p7.1), [§3.2](#S3.SS2.p8.1), [§3.4](#S3.SS4.p4.4), [§6](#S6.p2.3). - K. Meng, A. S. Sharma, A. J. Andonian, Y. Belinkov, and D. Bau (2023) Mass-editing memory in a transformer. In The Eleventh International Conference on Learning Representations, External Links: [Link](https://openreview.net/forum?id=MkbcAHIYgyS) Cited by: [§2](#S2.p7.1), [§5.1](#S5.SS1.p3.1), [§6](#S6.p2.3). - R. M. Nosofsky, T. J. Palmeri, and S. C. Mckinley (1994) Rule-plus-exception model of classification learning.. Psychological review 101 1, pp. 53–79. External Links: [Link](https://api.semanticscholar.org/CorpusID:6543807) Cited by: [§1](#S1.p3.1). - R. C. O’Reilly, R. Bhattacharyya, M. D. Howard, and N. Ketz (2014) Complementary learning systems. Cognitive Science 38 (6), pp. 1229–1248. External Links: [Document](https://dx.doi.org/https%3A//doi.org/10.1111/j.1551-6709.2011.01214.x), [Link](https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1551-6709.2011.01214.x), https://onlinelibrary.wiley.com/doi/pdf/10.1111/j.1551-6709.2011.01214.x Cited by: [§1](#S1.p3.1), [§2](#S2.p8.1). - O. Ovadia, M. Brief, M. Mishaeli, and O. Elisha (2024) Fine-tuning or retrieval? comparing knowledge injection in llms. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 237–250. External Links: [Link](https://api.semanticscholar.org/CorpusID:266162497) Cited by: [§3.2](#S3.SS2.p9.1), [§6](#S6.p1.1). - C. Packer, S. Wooders, K. Lin, V. Fang, S. G. Patil, I. Stoica, and J. E. Gonzalez (2024) MemGPT: towards llms as operating systems. External Links: 2310.08560, [Link](https://arxiv.org/abs/2310.08560) Cited by: [§1](#S1.p1.1), [§6](#S6.p1.1). - C. Packer (2025) Continual learning in token space. Note: Letta Research Blog[https://www.letta.com/blog/continual-learning](https://www.letta.com/blog/continual-learning) Cited by: [§3.3](#S3.SS3.p5.3). - J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein (2023) Generative agents: interactive simulacra of human behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. External Links: [Link](https://api.semanticscholar.org/CorpusID:258040990) Cited by: [§1](#S1.p1.1), [§6](#S6.p1.1). - N. Paulsen (2026) Context is what you need: the maximum effective context window for real world limits of llms. Advances in Artificial Intelligence and Machine Learning 06 (01), pp. 01–26. External Links: ISSN 2582-9793, [Link](http://dx.doi.org/10.54364/AAIML.2026.61268), [Document](https://dx.doi.org/10.54364/aaiml.2026.61268) Cited by: [Appendix A](#A1.p2.8), [§3.2](#S3.SS2.p10.2), [§4](#S4.p2.3). - J. H. Saltzer and M. D. Schroeder (1975) The protection of information in computer systems. Proceedings of the IEEE 63, pp. 1278–1308. External Links: [Link](https://api.semanticscholar.org/CorpusID:269166) Cited by: [§3.4](#S3.SS4.p1.1). - N. Shinn, F. Cassano, B. Labash, A. Gopinath, K. Narasimhan, and S. Yao (2023) Reflexion: language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems. External Links: [Link](https://api.semanticscholar.org/CorpusID:258833055) Cited by: [§1](#S1.p1.1), [§1](#S1.p2.1), [§6](#S6.p1.1). - Y. Sun, X. Li, K. Dalal, J. Xu, A. Vikram, G. Zhang, Y. Dubois, X. Chen, X. Wang, S. Koyejo, T. Hashimoto, and C. Guestrin (2025) Learning to (learn at test time): rnns with expressive hidden states. External Links: 2407.04620, [Link](https://arxiv.org/abs/2407.04620) Cited by: [Table 2](#A4.T2.8.10.1.5), [§5.1](#S5.SS1.p3.1). - V. N. Vapnik (1998) Statistical learning theory. Wiley. Cited by: [Appendix B](#A2.p12.3). - G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar (2023) Voyager: an open-ended embodied agent with large language models. External Links: 2305.16291, [Link](https://arxiv.org/abs/2305.16291) Cited by: [§1](#S1.p1.1), [§4](#S4.p5.7), [§6](#S6.p1.1). - H. Wang, G. Wang, H. Xiao, Y. Zhou, Y. Pan, J. Wang, K. Xu, Y. Wen, X. Ruan, X. Chen, and H. Qi (2026) Skill-sd: skill-conditioned self-distillation for multi-turn llm agents. External Links: 2604.10674, [Link](https://arxiv.org/abs/2604.10674) Cited by: [§5.1](#S5.SS1.p3.1), [§6](#S6.p2.3). - Y. Wu, T. Lin, Y. Zhou, F. Zhang, Q. Guo, X. Zhou, S. Wang, X. Liu, Y. Ma, and Y. Fang (2026) Memory in the LLM era: modular architectures and strategies in a unified framework. External Links: 2604.01707, [Link](https://arxiv.org/abs/2604.01707) Cited by: [§5.2](#S5.SS2.p1.1), [§6](#S6.p1.1). - W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang (2025) A-mem: agentic memory for llm agents. External Links: 2502.12110, [Link](https://arxiv.org/abs/2502.12110) Cited by: [§6](#S6.p1.1). - Z. Yang, Y. Song, I. Ahmed, and I. Harris (2026) Fine-tuning vs. rag for multi-hop question answering with novel knowledge. External Links: 2601.07054, [Link](https://arxiv.org/abs/2601.07054) Cited by: [§3.2](#S3.SS2.p9.1), [§6](#S6.p1.1). - T. Yao, Y. Chen, Y. Zheng, P. Li, Z. Shen, and K. Zhang (2026) ParamMem: augmenting language agents with parametric reflective memory. External Links: 2602.23320, [Link](https://arxiv.org/abs/2602.23320) Cited by: [§2](#S2.p7.1), [§3.1](#S3.SS1.p2.1), [§3.2](#S3.SS2.p9.1), [§6](#S6.p1.1). - Y. Yao, N. Zhang, Z. Xi, M. Wang, Z. Xu, S. Deng, and H. Chen (2024) Knowledge circuits in pretrained transformers. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, External Links: [Link](https://openreview.net/forum?id=YVXzZNxcag) Cited by: [§3.2](#S3.SS2.p8.1), [§5.2](#S5.SS2.p4.1). - J. Ye, Y. Yang, Y. Nan, S. Li, Q. Zhang, T. Gui, X. Huang, P. Wang, Z. Shi, and J. Fan (2025) Analyzing the effects of supervised fine-tuning on model knowledge from token and parameter levels. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 471–513. Cited by: [§3.2](#S3.SS2.p8.1), [§5.2](#S5.SS2.p4.1). - H. Yu, F. Zhu, G. Xie, and L. Shao (2026) Self-consolidation for self-evolving agents. External Links: 2602.01966, [Link](https://arxiv.org/abs/2602.01966) Cited by: [§6](#S6.p1.1). - Q. Zhan, Z. Liang, Z. Ying, and D. Kang (2024) InjecAgent: benchmarking indirect prompt injections in tool-integrated large language model agents. In Findings of the Association for Computational Linguistics: ACL 2024, L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, pp. 10471–10506. External Links: [Link](https://aclanthology.org/2024.findings-acl.624/), [Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.624) Cited by: [§3.4](#S3.SS4.p2.3), [§6](#S6.p2.3). - S. Zhang, J. Wang, R. Zhou, J. Liao, Y. Feng, Z. Li, Y. Zheng, W. Zhang, Y. Wen, Z. Li, F. Xiong, Y. Qi, B. Tang, and M. Wen (2026a) MemRL: self-evolving agents via runtime reinforcement learning on episodic memory. External Links: 2601.03192, [Link](https://arxiv.org/abs/2601.03192) Cited by: [§4](#S4.p6.7), [§6](#S6.p2.3), [Remark 2](#Thmtheorem2.p1.12.12). - X. Zhang, G. Wang, Y. Cui, W. Qiu, Z. Li, B. Zhu, and P. He (2026b) Experience compression spectrum: unifying memory, skills, and rules in llm agents. External Links: 2604.15877, [Link](https://arxiv.org/abs/2604.15877) Cited by: [§2](#S2.p3.3), [§4](#S4.p5.7), [§6](#S6.p2.3), [§7](#S7.p2.3). - Z. Zhang, X. Bo, C. Ma, R. Li, X. Chen, Q. Dai, J. Zhu, Z. Dong, and J. Wen (2024) A survey on the memory mechanism of large language model based agents. External Links: 2404.13501, [Link](https://arxiv.org/abs/2404.13501) Cited by: [§6](#S6.p1.1). - W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang (2024) MemoryBank: enhancing large language models with long-term memory. Proceedings of the AAAI Conference on Artificial Intelligence 38 (17), pp. 19724–19731. External Links: [Link](https://ojs.aaai.org/index.php/AAAI/article/view/29946), [Document](https://dx.doi.org/10.1609/aaai.v38i17.29946) Cited by: [§6](#S6.p1.1). - H. Zhou, Y. Chen, S. Guo, X. Yan, K. H. Lee, Z. Wang, K. Y. Lee, G. Zhang, K. Shao, L. Yang, and J. Wang (2025) Memento: fine-tuning llm agents without fine-tuning llms. External Links: 2508.16153, [Link](https://arxiv.org/abs/2508.16153) Cited by: [§4](#S4.p6.7), [§6](#S6.p2.3), [Remark 2](#Thmtheorem2.p1.12.12). - W. Zou, R. Geng, B. Wang, and J. Jia (2024) PoisonedRAG: knowledge corruption attacks to retrieval-augmented generation of large language models. In USENIX Security Symposium, External Links: [Link](https://api.semanticscholar.org/CorpusID:271854736) Cited by: [§3.4](#S3.SS4.p2.3), [§6](#S6.p2.3). ## Appendix A Proof of Theorem 1 (Performance Ceiling Bound) Setup. Let $\mathcal{T}_{m}$ be the class of $m$-hop chain reasoning tasks: given query $q=(e_{0},r_{1},\ldots,r_{m})$ and fact base $\mathcal{F}=\{(e_{i-1},r_{i},e_{i})\}_{i=1}^{m}$, the agent must return $e_{m}$. Retrieval upper bound. Retrieval inserts at most $K$ entries into context. For $m>K$, at least $m-K$ facts are unavailable. For $m\leq K$, Liu et al. [2023] show a U-shaped attention degradation: facts at context positions $p\in(1/4,3/4)$ suffer ${\sim}20\%$ relative performance degradation. Paulsen [2026] show effective context utilization saturates at ${\approx}20$k tokens even for 128k-token models. These establish an effective capacity ceiling $C_{R}K$, retrieval cannot solve $\mathcal{T}_{m}$; parametric memory can. The performance gap is monotonically increasing in $m$. $\square$ ## Appendix B Proof of Theorem 1 (Compositional Sample Complexity Separation) Full setup. Let $\mathcal{F}=\{f_{1},\ldots,f_{k}\}$ be the set of base concepts in a target domain, and $\oplus:\mathcal{F}\times\mathcal{F}\to\mathcal{Y}$ the domain-specific composition operator. Let $N=\binom{k}{2}$ denote the number of distinct unordered concept pairs. Both systems observe the same training data $D=\{(f_{i_{t}},f_{j_{t}},\oplus(f_{i_{t}},f_{j_{t}}))\}_{t=1}^{n}$, drawn uniformly from the $N$ possible pairs. Let $S\subseteq\binom{[k]}{2}$ be the set of pairs covered by $D$, with $|S|=n\leq N$. Formal definition of $\operatorname{CGC}$. $$ $\operatorname{CGC}(M,D)\;=\;\Pr_{(f_{i},f_{j})\sim\mathrm{Uniform}\bigl(\binom{\mathcal{F}}{2}\bigr)}\!\bigl[\,M(f_{i},f_{j})=\oplus(f_{i},f_{j})\,\bigr].$ $$ Part 1: Retrieval lower bound. For a retrieval-based system $M_{\mathrm{R}}$ with store $D$ and frozen model $f_{\theta_{\mathrm{frozen}}}$, consider a query pair $(f_{i},f_{j})$: - • If $(i,j)\in S$: the exact composition is stored and retrievable; accuracy is 1. - • If $(i,j)\notin S$: the system retrieves $K$ most relevant entries from $D$ and the frozen model processes them. This processing includes the LLM’s full inferential capacity: attention over retrieved examples, chain-of-thought reasoning, and any other in-context computation. By Assumption [3.2](#S3.SS2), accuracy on this query is at most $\bar{\alpha}<1$. Therefore: $$ $\displaystyle\operatorname{CGC}(M_{\mathrm{R}},D)$ $\displaystyle=\frac{n}{N}\cdot 1+\frac{N-n}{N}\cdot\alpha(K)$ $\displaystyle\leq\frac{n+(N-n)\bar{\alpha}}{N}.$ (1) $$ Setting $\operatorname{CGC}(M_{\mathrm{R}},D)\geq 1-\delta$ and solving for $n$: $$ $\displaystyle n+(N-n)\bar{\alpha}$ $\displaystyle\geq(1-\delta)N$ $\displaystyle n(1-\bar{\alpha})$ $\displaystyle\geq(1-\delta-\bar{\alpha})N$ $\displaystyle n\geq n_{R}$ $\displaystyle\triangleq\frac{1-\delta-\bar{\alpha}}{1-\bar{\alpha}}\cdot\binom{k}{2}.$ (2) $$ This bound requires $\delta<1-\bar{\alpha}$ (otherwise the frozen model alone achieves the target without any stored compositions). When $\bar{\alpha}$ is small (the frozen model cannot compose in this domain), $n_{R}\approx(1-\delta)\binom{k}{2}=\Theta(k^{2})$. Note on the strength of the retrieval model. The bound in ([1](#A2.E1)) is *not* an artefact of modeling the retrieval system as a naïve lookup table. The term $\alpha(K)$ explicitly accounts for the frozen model’s processing of retrieved content, including multi-step reasoning and in-context learning over the $K$ retrieved examples. The only assumption is that this processing cannot perfectly learn an out-of-distribution composition operator—i.e., $\alpha(K)\leq\bar{\alpha}<1$—which is an empirically testable claim about the frozen model’s ICL capacity on the target domain. Part 2: Parametric upper bound. A parametric system $M_{\mathrm{P}}$ fine-tunes on $D$ to learn $\oplus$ as a function $\hat{h}:\mathcal{F}\times\mathcal{F}\to\mathcal{Y}$ from a hypothesis class $\mathcal{H}$ with VC dimension $d$. By the fundamental theorem of statistical learning theory [Vapnik, 1998], for any distribution over $\mathcal{F}\times\mathcal{F}$ and any $\delta>0$, if $\oplus\in\mathcal{H}$ then $$ $n_{P}=O\!\left(\frac{d+\log(1/\delta)}{\delta}\right)$ $$ i.i.d. examples suffice for the empirical risk minimizer $\hat{h}$ to satisfy $\Pr[\hat{h}(f_{i},f_{j})\neq\oplus(f_{i},f_{j})]\leq\delta$, i.e. $\operatorname{CGC}(M_{\mathrm{P}},D)\geq 1-\delta$. Crucially, $n_{P}$ depends only on $d$ (the intrinsic complexity of $\oplus$) and $\delta$ (the target accuracy), *not* on $k$ (the number of base concepts). Part 3: Separation. The sample complexity ratio is: $$ $\frac{n_{R}}{n_{P}}=\frac{\frac{1-\delta-\bar{\alpha}}{1-\bar{\alpha}}\binom{k}{2}}{O\!\left(\frac{d+\log(1/\delta)}{\delta}\right)}=\Omega\!\left(\frac{k^{2}}{d}\right).$ $$ For structured operators with $d=O(k)$: $n_{R}/n_{P}=\Omega(k)$. For operators with $d=O(1)$ (e.g. parameterized group operations): $n_{R}/n_{P}=\Omega(k^{2})$. Part 4: Independence from context window size. Increasing $K$ (the number of retrieved examples) can only improve $\bar{\alpha}$, not eliminate the structural coverage requirement. For any $\bar{\alpha}<1$, the retrieval system still requires $n_{R}=\Omega(k^{2})$ stored compositions. The only way to reduce $n_{R}$ to $O(d)$ is to have $\bar{\alpha}\to 1$, which requires the frozen model to *already know* $\oplus$—in which case no learning of any kind is needed, and the distinction is moot. $\square$ ## Appendix C Proof of Proposition 1 (Information-Theoretic Bound on ICL Accuracy) Part 1: Unstructured operators. For $\mathcal{H}=\mathcal{Y}^{N}$ (the set of all functions from concept pairs to outputs) with a uniform prior, the values $\{\oplus(f_{i},f_{j})\}_{(i,j)\in\binom{[k]}{2}}$ are mutually independent and each uniformly distributed over $\mathcal{Y}$. Observing $K$ of these values on distinct pairs reveals nothing about the remaining $N-K$ values (by independence). The Bayes-optimal prediction accuracy on any unseen pair is therefore exactly $1/|\mathcal{Y}|$, so $\bar{\alpha}=1/|\mathcal{Y}|$. $\square$ Part 2: Structured operators (Fano bound). Let $V$ denote the identity of the true operator $\oplus\in\mathcal{H}$, drawn uniformly with $|\mathcal{H}|=M$. Let $Z=\{(f_{a_{t}},f_{b_{t}},\oplus(f_{a_{t}},f_{b_{t}}))\}_{t=1}^{K}$ be the $K$ observed examples. Since each observation takes values in $\mathcal{Y}$, the entropy of the observations is bounded: $H(Z)\leq K\log|\mathcal{Y}|$. By the data processing inequality, $I(V;Z)\leq H(Z)\leq K\log|\mathcal{Y}|$. Applying Fano’s inequality [Cover and Thomas, 2006] to identifying $V$ among $M$ equiprobable hypotheses: $$ $P(\hat{V}\neq V)\;\geq\;1-\frac{I(V;Z)+\log 2}{\log M}\;\geq\;1-\frac{K\log|\mathcal{Y}|+\log 2}{\log M}\;\triangleq\;1-\gamma.$ $$ Connecting identification failure to prediction error. Let $\mathcal{H}_{\mathrm{obs}}=\{h\in\mathcal{H}:h\text{ agrees with }\oplus\text{ on the }K\text{ observed pairs}\}$ be the set of operators consistent with the observations. Since the $K$ observations can produce at most $|\mathcal{Y}|^{K}$ distinct outcome sequences, at least $M/|\mathcal{Y}|^{K}$ operators survive the filter on average, so $|\mathcal{H}_{\mathrm{obs}}|\geq M/|\mathcal{Y}|^{K}$. These $|\mathcal{H}_{\mathrm{obs}}|$ operators are distinct elements of $\mathcal{H}$ that agree on $K$ pairs. They must therefore disagree on at least some of the remaining $N-K$ pairs. A counting argument gives the minimum number of disagreement pairs: each unseen pair can take at most $|\mathcal{Y}|$ distinct values across operators in $\mathcal{H}_{\mathrm{obs}}$, so representing $|\mathcal{H}_{\mathrm{obs}}|$ distinct profiles over the unseen pairs requires at least $$ $d_{\min}\;=\;\left\lceil\frac{\log|\mathcal{H}_{\mathrm{obs}}|}{\log|\mathcal{Y}|}\right\rceil\;\geq\;\frac{\log M-K\log|\mathcal{Y}|}{\log|\mathcal{Y}|}$ $$ unseen pairs on which the operators in $\mathcal{H}_{\mathrm{obs}}$ take at least two distinct values. On each such *disagreement pair*, the Bayes-optimal prediction accuracy is strictly below 1 (at most $1-1/|\mathcal{Y}|$, since at least two output values are represented in the posterior). On the remaining unseen pairs, all consistent operators agree and accuracy is 1. The average prediction accuracy on a uniformly random unseen pair is therefore: $$ $\bar{\alpha}\;\leq\;1-\frac{d_{\min}}{N-K}\cdot\frac{1}{|\mathcal{Y}|}\;\leq\;1-\frac{\log M-K\log|\mathcal{Y}|}{(N-K)\,|\mathcal{Y}|\,\log|\mathcal{Y}|}\;<\;1$ $$ whenever $\log M>K\log|\mathcal{Y}|$. $\square$ Interpretation. For the fully unstructured class ($M=|\mathcal{Y}|^{N}$), the bound gives $\bar{\alpha}\leq 1-(N-K)/((N-K)\cdot|\mathcal{Y}|)=1-1/|\mathcal{Y}|$, consistent with Part 1 (exact value $1/|\mathcal{Y}|$ is tighter). For a structured class with $M=|\mathcal{Y}|^{N^{\beta}}$ and $\beta<1$, $\bar{\alpha}<1$ whenever $K0$ on compositionally novel inputs. Note: Assumption [3](#Thmtheorem3) is implied by Assumption [3.2](#S3.SS2). If $\bar{\alpha}<1$ then $\varepsilon_{\mathrm{compose}}(\theta_{\mathrm{frozen}})\geq 1-\bar{\alpha}>0$. ###### Proposition 4 (Composition-error formulation) . Under Assumption [3](#Thmtheorem3), for any retrieval-based memory $M_{\mathrm{R}}$ with frozen model $\theta_{\mathrm{frozen}}$: $$ $\sup_{M_{\mathrm{R}}}\;\operatorname{Acc}(M_{\mathrm{R}},\,\mathcal{T}_{\mathrm{novel}})\;\leq\;1-\varepsilon_{\mathrm{compose}}(\theta_{\mathrm{frozen}})\;<\;1-\varepsilon_{\mathrm{compose}}(\theta^{*})\;\leq\;\sup_{M_{\mathrm{P}}}\;\operatorname{Acc}(M_{\mathrm{P}},\,\mathcal{T}_{\mathrm{novel}}),$ $$ where $\theta^{*}$ is the fine-tuned model and $\varepsilon_{\mathrm{compose}}(\theta^{*})<\varepsilon_{\mathrm{compose}}(\theta_{\mathrm{frozen}})$ by the PAC guarantee in Theorem [1](#Thmtheorem1). Relationship to Theorem [1](#Thmtheorem1). The composition-error formulation follows by setting $\bar{\alpha}=1-\varepsilon_{\mathrm{compose}}$ in Theorem [1](#Thmtheorem1). The sample-complexity formulation is strictly stronger: it quantifies *how much data* each system needs (the $\Omega(k^{2}/d)$ ratio), whereas the composition-error formulation establishes only the *direction* of the gap. ###### Assumption 3 (Domain specificity) . ###### Proposition 4 (Composition-error formulation) .