What Deserves Memory: Adaptive Memory Distillation for LLM Agents 

Report GitHub Issue

 × 

 Title: 

Content selection saved. Describe the issue below:

 Description: 

 Submit without GitHub 
 Submit in GitHub 

 Back to arXiv 

 Why HTML? 

 Report Issue 

 Back to Abstract 

 Download PDF 

 Abstract 

 1 Introduction 

 2 Related Works 

 2.1 Memory for LLM Agents 

 2.2 Cognitive Principles of Memory 

 3 Methodology 

 3.1 Overview & Motivations 

 The Structure Prior: Integrity of Episode. 

 The Representation Prior: Asymmetry of Perspective. 

 The Distillation Prior: Predictability Implies Redundancy. 

 3.2 Episodic Memory Integration 

 3.2.1 Local Message Partitioning 

 3.2.2 Narrative Episode Generation 

 Discussion. 

 3.2.3 Associative Memory Integration 

 3.3 Semantic Knowledge Distillation 

 3.3.1 Anticipatory Schema Synthesis 

 3.3.2 Prediction Error Distillation 

 3.3.3 Agnostic Knowledge Consolidation 

 3.4 Response Generation 

 4 Experiments 

 4.1 Experimental Setup 

 Datasets. 

 Baselines. 

 Evaluation Metrics. 

 Implementation Details. 

 4.2 Main Results (RQ1) 

 Strong Performance. 

 Exceptional Temporal Reasoning. 

 4.3 Efficiency Analysis (RQ2) 

 Memory Construction. 

 Response Generation. 

 4.4 Ablation Study (RQ3) 

 Prediction-error-based vs. Direct Distillation. 

 Native Management Contribution. 

 Episodic–Semantic Complementarity. 

 Observation Window Length. 

 4.5 Retrieval Hyperparameter Analysis (RQ4) 

 Top-K Sensitivity. 

 Index–Retrieve Strategy. 

 4.6 Third-Party Integration (RQ5) 

 4.7 Scalability Analysis (RQ6) 

 5 Conclusion 

 References 

 A Implementations of Management 

 A.1 Conceptual Model: Flat Summarization 

 Context Evocation. 

 Knowledge Consolidation. 

 A.2 Variant: Naive RAG 

 Context Evocation. 

 Knowledge Consolidation. 

 A.3 External Integration: Third-Party Systems 

 Context Evocation. 

 Knowledge Consolidation. 

 B Case Study 

 B.1 Temporal Reasoning 

 B.2 Open Domain 

 C Additional Experiment Results 

 Memory Construction Cost (Table  9 ) 

 Ablation Study (Table  10 ). 

 Observation Window Length (Table  11 ). 

 Top-K Sensitivity (Table  12 ). 

 Retrieval Strategy (Table  13 ). 

 Third-Party Management (Tables  14 and 15 ). 

 D Prompt Templates 

 D.1 Core Distillation Prompts 

 D.1.1 Local Message Partitioning Prompt ( 𝒫 par \mathcal{P}_{\text{par}} ) 

 D.1.2 Narrative Episode Generation Prompt ( 𝒫 nar \mathcal{P}_{\text{nar}} ) 

 D.1.3 Optimal Candidate Identification Prompt ( 𝒫 sel \mathcal{P}_{\text{sel}} ) 

 D.1.4 Episodic Integration Prompt ( 𝒫 int \mathcal{P}_{\text{int}} ) 

 D.1.5 Anticipatory Schema Synthesis Prompt ( 𝒫 ant \mathcal{P}_{\text{ant}} ) 

 D.1.6 Prediction Error Distillation Prompt ( 𝒫 dis \mathcal{P}_{\text{dis}} ) 

 D.1.7 Semantic Consolidation Prompt ( 𝒫 con \mathcal{P}_{\text{con}} ) 

 D.2 Direct Distillation Prompt ( Nemori -s) 

 D.3 Response Generation Prompt ( 𝒫 ans \mathcal{P}_{\text{ans}} ) 

 D.4 LLM-as-Judge Prompts 

 D.4.1 LoCoMo 

 D.4.2 LongMemEval S {}_{\text{S}} 

 License: CC BY 4.0

arXiv:2508.03341v4 [cs.AI] 16 Apr 2026

What Deserves Memory: Adaptive Memory Distillation for LLM Agents

 Wenquan Ma 1, 4 ,
 Jiayan Nan 2 1 1 footnotemark: 1 ,
 Wenlong Wu 3 ,
 Yize Chen 2 

 1 Fudan University  2 Shanda Group  3 Beihang University 

 4 Shanghai University of Finance and Economics

 wenquan.ma@outlook.com ,
 nanjiayan@shanda.com ,
 wlw@buaa.edu.cn 
 chenyize@shanda.com 
 Equal contribution. Corresponding author. 

Abstract

Memory systems for LLM agents struggle to determine what information deserves retention.
Existing approaches rely on predefined heuristics such as importance scores, emotional tags, or factual templates, encoding designer intuition rather than learning from the data itself.
Inspired by cognitive ideas, we propose Nemori , an adaptive memory distillation framework that casts the assessment of the experience’s future utility as a matter of predictability.
Specifically, Nemori comprises two cascading modules: Episodic Memory Integration transforms raw interactions into coherent narratives, and Semantic Knowledge Distillation extracts insights via prediction error.
Centering on distillation, the framework remains agnostic to downstream management.
Extensive experiments confirm that Nemori achieves strong performance, efficiency, and storage reduction.
Our work suggests that observing the intrinsic properties of interaction sequences offers a viable, data-driven alternative to heuristic-based memory design.
Code at: https://github.com/nemori-ai/nemori .

 What Deserves Memory: Adaptive Memory Distillation for LLM Agents 

Wenquan Ma 1, 4 † † thanks: Equal contribution. ,
Jiayan Nan 2 1 1 footnotemark: 1 † † thanks: Corresponding author. ,
Wenlong Wu 3 ,
Yize Chen 2 

 1 Fudan University  2 Shanda Group  3 Beihang University 

 4 Shanghai University of Finance and Economics 

 wenquan.ma@outlook.com ,
 nanjiayan@shanda.com ,
 wlw@buaa.edu.cn 
 chenyize@shanda.com 

 1 Introduction

The difficulty of maintaining long-term behavioral consistency in Large Language Model (LLM)-based agents stems from a fundamental conflict: the reliance of stateless LLMs on linearly expanding interaction trajectories versus the constraints of finite context window and the Lost in the Middle phenomenon (Liu et al. , 2024 ) .
Nevertheless, real-world applications, exemplified by personal assistants, autonomous agents and personalized recommendation systems, increasingly demand persistent interaction.
To address these challenges, memory systems that facilitate real-time context regulation have emerged as a viable and prevailing approach.

 Category 
 Method 

 Distillation 

 Management 

 Retrieval 

 Retrieval-time 
 Lewis et al. ( 2020 ) 

 — 

 — 

 Similarity search 

 Management-time 
 Packer et al. ( 2023 ) 

 — 

 Tiered storage 

 Function calls 

 Zhong et al. ( 2024 ) 

 — 

 Summary + forgetting 

 — 

 Kang et al. ( 2025 ) 

 — 

 Heat scoring 

 Two-step search 

 Li et al. ( 2025b ) 

 — 

 Hierarchical summary 

 Multi-step search 

 Anokhin et al. ( 2025 ) 

 — 

 Graph update 

 Graph spreading 

 Xu et al. ( 2025 ) 

 — 

 Adaptive note linking 

 — 

 Rasmussen et al. ( 2025 ) 

 — 

 Validity management 

 Reranking 

 Distillation-time 
 Park et al. ( 2023 ) 

 Importance scoring 

 Reflection trees 

 Weighted scoring 

 Huang et al. ( 2024 ) 

 Emotion tagging 

 — 

 Emotion matching 

 Chhikara et al. ( 2025 ) 

 Facts extraction 

 — 

 — 

 Li et al. ( 2025a ) 

 Summary + persona 

 — 

 Noun overlap 

 Pan et al. ( 2025 ) 

 Topic 

 Token compression 

 — 

 This paper 

 Prediction error 

 Agnostic 

 — 

 Table 1: Agent memory systems categorized by the stage at which memory utility is assessed. Retrieval-time methods defer assessment entirely; management-time methods filter post-hoc via access patterns; distillation-time methods assess at entry ingestion. Common practices omitted: raw retention in Distillation, conflict detection in Management, similarity search (non-graph) or graph traversal (graph-based) in Retrieval. Our approach assesses at distillation via prediction error rather than predefined heuristics. 

Memory systems identify useful experiences to facilitate future response generation through two stages: distillation , which determines the entry form of experiences, and management , which ensures their ongoing maintenance.
To this end, one category of approaches focuses on management by treating entries as opaque containers, where utility is inferred through observable structural metadata, such as access frequency (Kang et al. , 2025 ) , temporal decay (Zhong et al. , 2024 ) , or explicit relationships (Xu et al. , 2025 ) , foregoing the inspection of the nuanced content itself.
In contrast, another category of approaches intervenes during the initial distillation stage, selectively shaping the entry form.
This pre-positioning , while granting greater flexibility, must contend with future utility uncertainty.
Existing distillation methods typically address this by encoding designer intuition, such as importance scores (Park et al. , 2023 ) , emotional tags (Huang et al. , 2024 ) , or factual templates (Chhikara et al. , 2025 ) .
However, such heuristics risk introducing subjective bias, which is fatal during distillation as it can lead to irreversible information distortion, or causing systemic bloat, where the system tends to over-store to avoid such distortion, thereby amplifying retrieval noise.
This limitation necessitates an approach that assesses the potential utility grounded in the interaction experience itself.

Inspired by Predictive Coding Theory (Rao and Ballard, 1999 ; Friston, 2010 ; Clark, 2013 ) , we propose Nemori , a training-free framework that casts the assessment of experience utility as adaptive memory distillation over incoming observations that the agent fails to predict given existing knowledge, enabling a data-driven space.
As illustrated in Figure  1 , this framework, guided by three parsimonious priors over memory structure, representation and distillation, comprises two cascading modules, echoing Complementary Learning Systems (McClelland et al. , 1995 ) .
Specifically, the Episodic Memory Integration module first transforms raw interaction sequences into coherent episodic narratives.
The Semantic Knowledge Distillation module then extracts novel experience that existing knowledge cannot anticipate.
Centering on the distillation stage, Nemori remains agnostic to the underlying management, while a native management system is provided.
Our contributions:

1) Perspective. We formalize the distinction between distillation and management in memory construction, and derive priors from general data properties and cognitive ideas to guide distillation design.

2) Framework. We implement Nemori , a management-agnostic adaptive memory distillation framework, and equip it with a native management system.

3) Evaluation. We conduct extensive experiments demonstrating Nemori ’s strong performance, with pronounced advantages in longer context. When integrated with third-party management systems, Nemori enhances A-MEM and MemoryOS with 45–64% storage reduction while maintaining performance.

 2 Related Works

 2.1 Memory for LLM Agents

Agent memory systems decompose into three stages: distillation (what to retain), management (how to organize), and retrieval (how to surface content).
Unlike pure RAG (Lewis et al. , 2020 ) that defers judgment to query time, memory systems pre-position : enriching data with metadata at distillation, then utilizing it for management and retrieval.
Table  1 categorizes works by when enrichment occurs.
 Management-time methods enrich post-hoc via decay weights, tiered storage, access frequency, or relationship linking (Zhong et al. , 2024 ; Packer et al. , 2023 ; Kang et al. , 2025 ; Xu et al. , 2025 ) .
 Distillation-time methods enrich at ingestion through importance scoring, emotional tagging, or fact extraction (Park et al. , 2023 ; Huang et al. , 2024 ; Chhikara et al. , 2025 ) .

 Figure 1: Overview of the Nemori framework. The system comprises two cascading modules guided by three priors: Episodic Memory Integration (top) transforms raw interactions into coherent narrative episodes, and Semantic Knowledge Distillation (bottom) extracts insights via prediction error. The framework can serve as a distillation layer complement native or third-party management systems (right). 

 2.2 Cognitive Principles of Memory

Predictive Coding Theory (Rao and Ballard, 1999 ) , originally from visual neuroscience, posits that higher cortical areas send predictions downward while lower areas propagate primarily the residual prediction error upward.
 Friston ( 2010 ) generalized this into the Free Energy Principle, a unifying framework across perception, action, and learning.
 Clark ( 2013 ) further extended it, arguing that brains are fundamentally prediction machines.
 Nemori adapts this insight to agent memory design: prediction error signals information worth retaining; what is predictable is therefore redundant.

 3 Methodology

 Nemori is an adaptive memory distillation framework inspired by cognitive ideas (McClelland et al. , 1995 ; Rao and Ballard, 1999 ; Friston, 2010 ; Clark, 2013 ) .
This management-agnostic framework can serve as a distillation layer that complements either native or third-party memory systems like A-MEM or MemoryOS.
A production-grade implementation is provided at https://github.com/nemori-ai/nemori .

 3.1 Overview & Motivations

As illustrated in Figure  1 , Nemori comprises two cascading modules guided by three priors as inductive biases. These priors capture the parsimonious features of continuous interaction sequences, establishing a plastic, data-driven environment in which intrinsic dynamics drive the partitioning, representation, and distillation of experience into memory.

The Structure Prior: Integrity of Episode.

Interaction sequences exhibit natural grouping.
Interactions within each episodic group are mutually contextualizing: individual messages derive their meaning, partly, from surrounding ones, and finer-grained or arbitrary fragmentation would sever the context that renders them interpretable.

This prior requires the framework to define episodes respecting latent integrity among interactions, rather than imposing heuristic chunking.

The Representation Prior: Asymmetry of Perspective.

Memory serves recall.
Recalling is essentially a form of reasoning, an allocentric reconstruction of events, whereas raw episodes are egocentric and inherently noisy.

This prior requires the framework to transform raw episodes into narrative representations that highlight logical structures while preserving salient details, bridging the gap between chaotic perception and rational retrieval.

The Distillation Prior: Predictability Implies Redundancy.

Information within interaction sequences is highly redundant.
From the perspective of predictive coding, the unexpected information is a natural candidate for memory consolidation.

This key prior requires the framework to distill memory by inspecting the semantic differential between the actual interactions and their anticipatory schema derived from existing knowledge.

In the following sections, we detail the framework implementation guided by these priors.

 3.2 Episodic Memory Integration

Guided by the structure prior and the representation prior, this module integrates raw interactions into episodic memories and prepares them for subsequent distillation.
It is further divided into three submodules: Local Message Partitioning, Narrative Episode Generation and Associative Memory Integration.

 3.2.1 Local Message Partitioning

Guided by the structure prior, this submodule resolves the continuous interaction sequence within an observation window into a discrete partition.
We model the interactions between an agent and its environment as a sequence of message exchanges, maintaining a dedicated message buffer ℬ \mathcal{B} .
At any time t t , the buffer state is represented as a queue of messages ℬ t = { m 1 , m 2 , … , m z } \mathcal{B}_{t}=\{m_{1},m_{2},\dots,m_{z}\} , where each message m i = ( r i , c i , τ i ) m_{i}=(r_{i},c_{i},\tau_{i}) specifies the sender, content and timestamp, respectively.
New interactions are appended to the rear of ℬ t \mathcal{B}_{t} as they occur.

The partitioning process is triggered once the buffer size | ℬ t | |\mathcal{B}_{t}| reaches a predefined observation window length w ∈ ℤ + w\in\mathbb{Z}^{+} .
At this juncture, the submodule performs a partitioning operation:

 𝐎 ← f LLM ​ ( 𝒫 par ∥ ℬ t ) , \mathbf{O}\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{par}}\parallel\mathcal{B}_{t}), 

where 𝒫 par \mathcal{P}_{\text{par}} is a prompt that instructs the LLM to discern the latent integrity and local nuances within the window and partition the messages accordingly.
The output 𝐎 = { O 1 , O 2 , … , O n } \mathbf{O}=\{O_{1},O_{2},\dots,O_{n}\} (where n ≤ w n\leq w ) constitutes a partition of the index set { 1 , 2 , … , w } \{1,2,\dots,w\} .
Specifically, the O j O_{j} are pairwise disjoint and their union covers the index set.
The submodule then maps these indices back to the message buffer to form a collection of raw episodes 𝐏 = { P 1 , … , P n } \mathbf{P}=\{P_{1},\dots,P_{n}\} , where each P j P_{j} is the subsequence of ℬ t \mathcal{B}_{t} indexed by O j O_{j} .

Finally, 𝐏 \mathbf{P} is transferred to the Narrative Episode Generation submodule, and the buffer ℬ \mathcal{B} is reset to empty to await subsequent incoming messages.

 3.2.2 Narrative Episode Generation

Guided by the representation prior, this submodule transforms the received raw episodes into narrative representations.
For each raw episode P j ∈ 𝐏 P_{j}\in\mathbf{P} , the submodule generates a narrative episode N j N_{j} and a corresponding episodic cue c j c_{j} tailored for semantic distillation:

 ( N j , c j ) ← f LLM ​ ( 𝒫 nar ∥ P j ) , (N_{j},c_{j})\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{nar}}\parallel P_{j}), 

where 𝒫 nar \mathcal{P}_{\text{nar}} instructs the LLM to highlight the logical structure and constituent elements within the interaction.

Subsequently, the submodule computes an embedding index 𝐯 j \mathbf{v}_{j} to enable associative retrieval:

 𝐯 j ← f emb ​ ( c j ∥ N j ) , \mathbf{v}_{j}\leftarrow f_{\text{emb}}(c_{j}\parallel N_{j}), 

where f emb f_{\text{emb}} denotes the embedding model.

Each episodic memory is represented as M j = ( c j , N j , P j , 𝐯 j ) M_{j}=(c_{j},N_{j},P_{j},\mathbf{v}_{j}) . Finally, the collection { M j } \{M_{j}\} is transferred to the Associative Memory Integration submodule.

Discussion.

Two aspects of this design merit attention. First, it enables dual-mode retrieval: returning N N directly for efficiency, or returning raw P P for precision-critical domains. Second, from this point onward, the episode becomes the primary processing unit throughout the pipeline, avoiding the message-wise processing that many baselines fall into and that incurs substantial cost overhead (see Section  4.3 ).

 3.2.3 Associative Memory Integration

This submodule dynamically integrates episodes that may have been sundered by the constraints of the observation window length.
For each new episodic memory M j M_{j} , the submodule performs an integration check against the existing episodic database 𝒟 e \mathcal{D}_{e} .
It first retrieves the top K e K_{e} candidates based on cosine similarity:

 𝐂 = { U 1 , U 2 , … , U K e } ← Search ⁡ ( 𝒟 e , 𝐯 j , K e ) , \mathbf{C}=\{U_{1},U_{2},\dots,U_{K_{e}}\}\leftarrow\operatorname{Search}(\mathcal{D}_{e},\mathbf{v}_{j},K_{e}), 

where each candidate U k = ( c k , N k , P k , 𝐯 k ) U_{k}=(c_{k},N_{k},P_{k},\mathbf{v}_{k}) .

The submodule then selects the optimal integration target:

 i ​ d ​ x ← f LLM ​ ( 𝒫 sel ​ ‖ ( c j , N j ) ‖ ​ { ( c k , N k ) } k = 1 K e ) , idx\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{sel}}\parallel(c_{j},N_{j})\parallel\{(c_{k},N_{k})\}_{k=1}^{K_{e}}), 

where 𝒫 sel \mathcal{P}_{\text{sel}} instructs the LLM to identify the target candidate that shares episodic continuity with the new memory.
The output i ​ d ​ x ∈ { 1 , … , K e } ∪ { − 1 } idx\in\{1,\dots,K_{e}\}\cup\{-1\} determines the subsequent operation:

 Case 1 ( i ​ d ​ x = k idx=k ): The LLM integrates two memories: ( c ν , N ν ) ← f LLM ​ ( 𝒫 int ​ ‖ c k ‖ ​ N k ​ ‖ c j ‖ ​ N j ) (c_{\nu},N_{\nu})\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{int}}\parallel c_{k}\parallel N_{k}\parallel c_{j}\parallel N_{j}) , superseding U k U_{k} with M ν = ( c ν , N ν , P k ∥ P j , f emb ( c ν ∥ N ν ) ) M_{\nu}=(c_{\nu},N_{\nu},P_{k}\parallel P_{j},f_{\text{emb}}(c_{\nu}\parallel N_{\nu})) .

 Case 2 ( i ​ d ​ x = − 1 idx=-1 ): No continuity found; M j M_{j} is inserted as a distinct entry.

Finally, the resulting episodic memory ( M ν M_{\nu} or M j 

... [OUTPUT TRUNCATED - 30000 chars omitted out of 80000 total] ...

 Document 

 Cited by: §1 .

 A. Maharana, D. Lee, S. Tulyakov, M. Bansal, F. Barbieri, and Y. Fang (2024) 
 Evaluating very long-term conversational memory of LLM agents .

 In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , L. Ku, A. Martins, and V. Srikumar (Eds.) ,

 Bangkok, Thailand , pp. 13851–13870 .

 External Links: Link ,
 Document 

 Cited by: §4.1 .

 J. L. McClelland, B. L. McNaughton, and R. C. O’Reilly (1995) 
 Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. .

 Psychological review 102 ( 3 ), pp. 419 .

 Cited by: §1 ,
 §3 .

 C. Packer, V. Fang, S. G. Patil, K. Lin, S. Wooders, and J. E. Gonzalez (2023) 
 MemGPT: towards llms as operating systems .

 CoRR abs/2310.08560 .

 External Links: Link ,
 Document ,
 2310.08560 

 Cited by: Table 1 ,
 §2.1 .

 Z. Pan, Q. Wu, H. Jiang, X. Luo, H. Cheng, D. Li, Y. Yang, C. Lin, H. V. Zhao, L. Qiu, and J. Gao (2025) 
 SeCom: on memory construction and retrieval for personalized conversational agents .

 In The Thirteenth International Conference on Learning Representations,
ICLR 2025, Singapore, April 24-28, 2025 ,

 External Links: Link 

 Cited by: Table 1 .

 J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein (2023) 
 Generative agents: interactive simulacra of human behavior .

 In Proceedings of the 36th Annual ACM Symposium on User Interface Software
and Technology, UIST 2023, San Francisco, CA, USA, 29 October 2023-
1 November 2023 , S. Follmer, J. Han, J. Steimle, and N. H. Riche (Eds.) ,

 pp. 2:1–2:22 .

 External Links: Link ,
 Document 

 Cited by: Table 1 ,
 §1 ,
 §2.1 .

 R. P. Rao and D. H. Ballard (1999) 
 Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects .

 Nature neuroscience 2 ( 1 ), pp. 79–87 .

 Cited by: §1 ,
 §2.2 ,
 §3 .

 P. Rasmussen, P. Paliychuk, T. Beauvais, J. Ryan, and D. Chalef (2025) 
 Zep: A temporal knowledge graph architecture for agent memory .

 CoRR abs/2501.13956 .

 External Links: Link ,
 Document ,
 2501.13956 

 Cited by: Table 1 ,
 §4.1 ,
 §4.1 .

 D. Wu, H. Wang, W. Yu, Y. Zhang, K. Chang, and D. Yu (2025) 
 LongMemEval: benchmarking chat assistants on long-term interactive memory .

 In The Thirteenth International Conference on Learning Representations,
ICLR 2025, Singapore, April 24-28, 2025 ,

 External Links: Link 

 Cited by: §4.1 .

 W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang (2025) 
 A-mem: agentic memory for LLM agents .

 In The Thirty-ninth Annual Conference on Neural Information Processing Systems ,

 External Links: Link 

 Cited by: §A.3 ,
 Table 1 ,
 §1 ,
 §2.1 ,
 §4.1 .

 W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang (2024) 
 MemoryBank: enhancing large language models with long-term memory .

 In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI
2024, Thirty-Sixth Conference on Innovative Applications of Artificial
Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances
in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver,
Canada , M. J. Wooldridge, J. G. Dy, and S. Natarajan (Eds.) ,

 pp. 19724–19731 .

 External Links: Link ,
 Document 

 Cited by: Table 1 ,
 §1 ,
 §2.1 .

 Appendix A Implementations of Management

This appendix details three instantiations of the management interfaces defined in Section  3.3 , demonstrating Nemori ’s architectural flexibility.

 A.1 Conceptual Model: Flat Summarization

This configuration only serves as a conceptual illustration to aid understanding. Here, ℳ \mathcal{M} maintains a single monolithic summary rather than a structured database.

Context Evocation.

The global summary 𝒮 s ​ u ​ m \mathcal{S}_{sum} is returned as constant context:

 Evoke ( M i ​ n , ℳ ) : 𝒮 i ​ n ← 𝒮 s ​ u ​ m \operatorname{Evoke}(M_{in},\mathcal{M}):\quad\mathcal{S}_{in}\leftarrow\mathcal{S}_{sum} 

Knowledge Consolidation.

New insights are directly merged into the summary:

 𝒮 s ​ u ​ m ← f LLM ​ ( 𝒫 sum ​ ‖ 𝒦 i ​ n ‖ ​ 𝒮 s ​ u ​ m ) \mathcal{S}_{sum}\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{sum}}\parallel\mathcal{K}_{in}\parallel\mathcal{S}_{sum}) 

where 𝒫 sum \mathcal{P}_{\text{sum}} instructs the LLM to merge distilled insights into the summary.

 A.2 Variant: Naive RAG

This configuration serves as an ablation in Section  4.4 , applying no management to semantic memory. Distilled insights are directly stored and retrieved via similarity search, without conflict resolution.

Context Evocation.

For each input M i ​ n M_{in} , the interface retrieves top- K s K_{s} semantically similar entries:

 𝒮 i ​ n ← Top − ⁡ K s ​ ( S r ∈ 𝒟 s ​ ∣ sim ⁡ ( 𝐯 i ​ n , 𝐮 r ) > ​ τ ) \mathcal{S}_{in}\leftarrow\operatorname{Top-}K_{s}\left(S_{r}\in\mathcal{D}_{s}\mid\operatorname{sim}(\mathbf{v}_{in},\mathbf{u}_{r})>\tau\right) 

Knowledge Consolidation.

Each distilled insight k q ∈ 𝒦 i ​ n k_{q}\in\mathcal{K}_{in} is simply embedded and appended:

 𝒟 s ← 𝒟 s ∪ { ( k q , f emb ​ ( k q ) ) } \mathcal{D}_{s}\leftarrow\mathcal{D}_{s}\cup\{(k_{q},f_{\text{emb}}(k_{q}))\} 

No management like conflict detection or merging is performed. This contrasts with our native implementation (Section  3.3 ), which applies the full consolidation logic with new/merge/conflict resolution.

 A.3 External Integration: Third-Party Systems

 Nemori can be implemented with third-party management systems (Xu et al. , 2025 ; Kang et al. , 2025 ) by intercepting their context assembly and injecting distilled content.

Context Evocation.

Most of the memory systems fundamentally operate by conditioning response generation on related context.
We intercept this context buffer ℬ ~ \tilde{\mathcal{B}} , assembled by the host’s management logic for query M i ​ n M_{in} , and repurpose it as the basis for prediction: instead of generating a response, we use it to synthesize an anticipatory schema of what should have occurred:

 𝒮 i ​ n ← ℬ ~ \mathcal{S}_{in}\leftarrow\tilde{\mathcal{B}} 

Knowledge Consolidation.

Each distilled insight k q ∈ 𝒦 i ​ n k_{q}\in\mathcal{K}_{in} is injected as an independent message into the host’s input sequence, allowing the external system to manage it natively.
Notably, most of the memory systems discussed in this paper are designed to process explicit factual knowledge, making Nemori ’s distilled semantic memory a suitable input.

 Algorithm 1 Nemori Memory Distillation 

 1: Message buffer ℬ t = { m 1 , … , m z } \mathcal{B}_{t}=\{m_{1},\dots,m_{z}\} 

 2: Updated episodic database 𝒟 e \mathcal{D}_{e} , semantic database 𝒟 s \mathcal{D}_{s} 

 3: % — Episodic Memory Integration (§ 3.2 ) — 

 4: Partition ℬ t \mathcal{B}_{t} into raw episodes 𝐏 = { P 1 , … , P n } ← f LLM ​ ( 𝒫 par ∥ ℬ t ) \mathbf{P}=\{P_{1},\dots,\allowbreak P_{n}\}\leftarrow f_{\text{LLM}}(\allowbreak\mathcal{P}_{\text{par}}\parallel\mathcal{B}_{t}) 

 5: for each raw episode P j ∈ 𝐏 P_{j}\in\mathbf{P} do 

 6:   Generate narrative and cue ( N j , c j ) ← f LLM ​ ( 𝒫 nar ∥ P j ) (N_{j},c_{j})\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{nar}}\parallel P_{j}) 

 7:   Compute embedding 𝐯 j ← f emb ​ ( c j ∥ N j ) \mathbf{v}_{j}\leftarrow f_{\text{emb}}(c_{j}\parallel N_{j}) 

 8:   Retrieve candidates from 𝒟 e \mathcal{D}_{e} and decide merge-or-insert

 9:   Obtain episodic memory M i ​ n M_{in} (merged M ν M_{\nu} or new M j M_{j} )

 10: % — Semantic Knowledge Distillation (§ 3.3 ) — 

 11:   Evoke context 𝒮 i ​ n ← Evoke ⁡ ( M i ​ n , ℳ ) \mathcal{S}_{in}\leftarrow\operatorname{Evoke}(M_{in},\mathcal{M}) 

 12:   Synthesize anticipatory schema P ^ i ​ n ← f LLM ​ ( 𝒫 ant ​ ‖ c i ​ n ‖ ​ 𝒮 i ​ n ) \hat{P}_{in}\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{ant}}\parallel c_{in}\parallel\mathcal{S}_{in}) 

 13:   Distill semantic insights 𝒦 i ​ n ← f LLM ​ ( 𝒫 dis ​ ‖ P i ​ n ‖ ​ P ^ i ​ n ) \mathcal{K}_{in}\leftarrow f_{\text{LLM}}(\allowbreak\mathcal{P}_{\text{dis}}\allowbreak\parallel P_{in}\parallel\hat{P}_{in}) 

 14:    Consolidate ⁡ ( 𝒦 i ​ n , ℳ ) \operatorname{Consolidate}(\mathcal{K}_{in},\mathcal{M}) 

 15: end for 

 Algorithm 2 Nemori Response Generation 

 1: Query Q Q , episodic database 𝒟 e \mathcal{D}_{e} , semantic database 𝒟 s \mathcal{D}_{s} 

 2: Response a a 

 3: Compute query embedding 𝐯 Q ← f emb ​ ( Q ) \mathbf{v}_{Q}\leftarrow f_{\text{emb}}(Q) 

 4: Retrieve ℛ ~ e ← Search ⁡ ( 𝒟 e , 𝐯 Q , k ) \tilde{\mathcal{R}}_{e}\leftarrow\operatorname{Search}(\mathcal{D}_{e},\mathbf{v}_{Q},k) ; extract ℛ e = { N i } i = 1 k \mathcal{R}_{e}=\{N_{i}\}_{i=1}^{k} , ℛ p = { P d } d = 1 r \mathcal{R}_{p}=\{P_{d}\}_{d=1}^{r} 

 5: Retrieve ℛ ~ s ← Search ⁡ ( 𝒟 s , 𝐯 Q , m ) \tilde{\mathcal{R}}_{s}\leftarrow\operatorname{Search}(\mathcal{D}_{s},\mathbf{v}_{Q},m) ; extract ℛ s = { s j } j = 1 m \mathcal{R}_{s}=\{s_{j}\}_{j=1}^{m} 

 6: Generate response a ← f LLM ​ ( 𝒫 ans ​ ‖ Q ‖ ​ ℛ e ​ ‖ ℛ p ‖ ​ ℛ s ) a\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{ans}}\parallel Q\parallel\mathcal{R}_{e}\parallel\mathcal{R}_{p}\parallel\mathcal{R}_{s}) 

 7: return a a 

 Appendix B Case Study

This section presents two representative cases from the main text, highlighting how Nemori supports temporal reasoning and open-domain question answering.

 B.1 Temporal Reasoning

To illustrate how Nemori enhances response quality, we provide a representative case from the LoCoMo dataset.

 Question: “When did Jon receive mentorship?” 

 Challenge: The original conversation contains relative temporal references like “yesterday” without explicit dates, requiring temporal reasoning.

 Full Context baseline: Confused by the term “yesterday” in the raw dialogue, the model incorrectly answered with the conversation date (June 16).

 Nemori : Retrieved both the relevant episodic memory (preserving conversational context) and a semantic memory that had already distilled the temporal information into explicit fact: “Jon was mentored on June 15, 2023.” By combining episodic context with pre-reasoned semantic knowledge, Nemori transforms complex reasoning into simple fact retrieval.

 Insight: This demonstrates the capability of “reasoning during memory formation.” The prediction error highlight that the specific date is unexpected given prior knowledge, prompting its distillation as semantic memory.

 B.2 Open Domain

On the Open Domain subset, Nemori ’s LLM score is slightly below the strongest memory system baseline, with gaps of 6.8% under gpt-4.1-mini (56.3 vs. 60.4) and 3.8% under gpt-4o-mini (45.8 vs. 47.6). We note that this subset is not a pure measure of the memory procedure’s effectiveness, specifically:

In LoCoMo, many such questions are not directly answerable from the original conversation history alone; instead, they require the backbone model to recognize a conversational description and map it to an item of general world knowledge. As a result, performance in this category depends not only on memory quality, but also on the model’s prior knowledge.

A representative example is the question: “What is the game with different colored cards that John was talking about with James?” The gold answer is “UNO” , but the dialogue itself never explicitly names UNO. Instead, the transcript only states that the players discussed a game with multi-colored cards and matching by color or number, while also noting that the speaker had forgotten its name. Accordingly, Nemori ’s episodic memory preserves this conversational evidence, and the semantic memory distills the same game description, but neither memory can inject the missing lexical label if it is absent from the interaction history. In such cases, whether the final answer becomes “UNO” depends largely on the backbone model’s ability to recognize the description from prior knowledge, rather than on a failure of memory distillation or retrieval.

 Appendix C Additional Experiment Results

This appendix provides detailed experimental results that supplement the main paper. All experiments use the setup described in Section  4.1 .

 Component 
 Input (k) 
 Output (k) 
 Total (k) 
 Ratio 

 Partition (§ 3.2.1 ) 
 44.7 
 4.6 
 49.3 
 15.3% 

 Narration (§ 3.2.2 ) 
 99.8 
 23.9 
 123.6 
 38.3% 

 Integration (§ 3.2.3 ) 
 43.8 
 8.4 
 52.2 
 16.2% 

 Distillation (§ 3.3 ) 
 88.9 
 8.8 
 97.7 
 30.3% 

 Table 9: Component-wise breakdown of Nemori ’s memory construction cost on LoCoMo with gpt-4o-mini. 

 Configuration 
 Mgmt 
 Temporal Reasoning 
 Open Domain 
 Multi-Hop 
 Single-Hop 
 Overall 

 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 

 gpt-4o-mini 

 Nemori-s 
 ✓ 
 33.3 
 36.8 
 31.1 
 49.0 
 24.4 
 18.6 
 47.9 
 30.5 
 20.2 
 60.3 
 39.7 
 32.1 
 51.7 
 36.4 
 28.9 

 ✗ 
 32.7 
 35.9 
 30.4 
 40.6 
 21.8 
 17.0 
 47.5 
 31.3 
 20.7 
 62.1 
 40.3 
 32.7 
 52.0 
 36.6 
 29.1 

 w/o e 
 ✓ 
 57.9 
 53.0 
 44.6 
 53.1 
 26.5 
 19.6 
 57.8 
 35.5 
 24.4 
 70.8 
 49.4 
 40.4 
 64.6 
 46.2 
 37.1 

 ✗ 
 56.7 
 52.8 
 44.8 
 54.2 
 27.8 
 20.6 
 59.9 
 36.5 
 24.8 
 71.1 
 48.9 
 39.8 
 65.0 
 46.2 
 36.9 

 w/o s 
 ✓ 
 32.7 
 38.9 
 33.0 
 42.7 
 22.2 
 17.0 
 53.9 
 33.0 
 22.2 
 64.7 
 44.1 
 36.1 
 54.7 
 39.6 
 31.7 

 w/o p 
 ✓ 
 56.7 
 52.7 
 43.4 
 45.8 
 25.0 
 19.3 
 59.9 
 36.3 
 23.5 
 77.5 
 51.7 
 40.7 
 68.0 
 47.4 
 36.8 

 Nemori 
 ✓ 
 67.6 
 57.3 
 47.6 
 45.8 
 23.9 
 18.5 
 61.7 
 38.1 
 26.0 
 81.9 
 54.8 
 43.8 
 73.0 
 50.3 
 39.7 

 gpt-4.1-mini 

 Nemori-s 
 ✓ 
 46.4 
 42.2 
 33.7 
 49.0 
 26.5 
 20.7 
 67.4 
 36.2 
 28.8 
 74.9 
 44.5 
 39.0 
 66.0 
 41.4 
 34.9 

 ✗ 
 47.0 
 42.5 
 32.5 
 50.0 
 28.5 
 22.7 
 70.6 
 38.9 
 29.9 
 72.7 
 42.8 
 37.4 
 65.5 
 41.1 
 34.1 

 w/o e 
 ✓ 
 63.2 
 49.6 
 41.1 
 52.1 
 27.2 
 21.2 
 72.7 
 38.9 
 29.1 
 82.4 
 53.2 
 47.0 
 74.7 
 48.2 
 40.9 

 ✗ 
 65.4 
 51.0 
 42.5 
 56.3 
 29.1 
 23.0 
 70.2 
 39.0 
 29.5 
 82.2 
 52.2 
 45.8 
 74.9 
 48.1 
 40.7 

 w/o s 
 ✓ 
 73.5 
 54.3 
 46.9 
 55.2 
 26.9 
 21.3 
 73.1 
 41.9 
 32.5 
 81.9 
 53.7 
 47.3 
 76.9 
 50.0 
 42.9 

 w/o p 
 ✓ 
 67.3 
 53.4 
 45.0 
 52.1 
 24.7 
 19.5 
 71.3 
 40.1 
 30.7 
 83.1 
 51.4 
 45.2 
 75.7 
 48.1 
 40.9 

 Nemori 
 ✓ 
 77.3 
 58.7 
 50.7 
 56.3 
 31.7 
 25.1 
 74.8 
 40.8 
 31.7 
 87.0 
 55.7 
 49.5 
 80.8 
 52.1 
 45.0 

 Table 10: Category-wise Ablation study on LoCoMo. Nemori-s = semantic-only (direct distillation); w/o e = without episodic retrieval; w/o s = without semantic retrieval; w/o p = without adaptive partitioning (fixed 20-message chunks); Nemori = full framework. Mgmt: ✓ = with native management, ✗ = naive RAG detailed in Section  A.2 . 

 w w 
 Temporal 
 Open Domain 
 Multi-Hop 
 Single-Hop 
 Overall 

 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 

 5 
 77.0 
 57.6 
 49.6 
 59.4 
 30.0 
 24.9 
 73.8 
 43.0 
 34.0 
 86.3 
 55.0 
 48.5 
 80.4 
 51.8 
 44.6 

 10 
 77.6 
 59.0 
 50.4 
 57.3 
 29.5 
 24.7 
 77.0 
 44.0 
 34.3 
 85.7 
 54.5 
 48.1 
 80.7 
 52.0 
 44.6 

 20 
 77.3 
 58.7 
 50.7 
 56.3 
 31.7 
 25.1 
 74.8 
 40.8 
 31.7 
 87.0 
 55.7 
 49.5 
 80.8 
 52.1 
 45.0 

 30 
 76.6 
 57.7 
 49.6 
 60.4 
 32.2 
 26.1 
 79.4 
 45.0 
 34.8 
 86.0 
 55.4 
 48.8 
 81.2 
 52.5 
 45.0 

 40 
 76.3 
 58.6 
 50.4 
 54.2 
 27.1 
 21.4 
 77.3 
 42.8 
 34.0 
 86.4 
 55.0 
 48.4 
 80.7 
 51.8 
 44.5 

 Table 11: Performance across different observation window lengths on LoCoMo dataset with gpt-4.1-mini. w=20 (bold) is the default setting used in main experiments. 

 k k 
 Temporal 
 Open Domain 
 Multi-Hop 
 Single-Hop 
 Overall 

 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 

 gpt-4o-mini 

 2 
 62.3 
 55.3 
 46.5 
 41.7 
 20.9 
 15.4 
 55.0 
 33.7 
 21.8 
 75.0 
 51.1 
 40.9 
 66.6 
 46.9 
 37.0 

 5 
 64.5 
 56.7 
 47.4 
 47.9 
 24.9 
 19.2 
 62.4 
 36.7 
 25.0 
 79.1 
 53.3 
 42.4 
 71.0 
 49.2 
 38.8 

 10 
 67.6 
 57.3 
 47.6 
 45.8 
 23.9 
 18.5 
 61.7 
 38.1 
 26.0 
 81.9 
 54.8 
 43.8 
 73.0 
 50.3 
 39.7 

 15 
 68.9 
 58.6 
 48.4 
 44.8 
 24.5 
 19.0 
 61.4 
 37.0 
 25.4 
 83.1 
 54.9 
 43.6 
 73.8 
 50.5 
 39.8 

 20 
 67.9 
 57.4 
 47.7 
 45.8 
 24.3 
 18.9 
 63.5 
 38.2 
 25.9 
 82.8 
 54.7 
 43.1 
 73.8 
 50.4 
 39.4 

 30 
 68.2 
 58.9 
 48.4 
 45.8 
 23.9 
 18.8 
 64.5 
 37.7 
 25.6 
 83.2 
 55.0 
 43.3 
 74.4 
 50.7 
 39.6 

 gpt-4.1-mini 

 2 
 68.5 
 52.8 
 45.5 
 52.1 
 25.7 
 20.5 
 64.5 
 38.2 
 28.3 
 80.6 
 51.7 
 45.6 
 73.4 
 47.8 
 40.9 

 5 
 74.1 
 56.7 
 48.9 
 55.2 
 28.8 
 22.6 
 73.1 
 41.9 
 32.3 
 86.1 
 54.4 
 48.1 
 79.3 
 51.0 
 43.8 

 10 
 77.3 
 58.7 
 50.7 
 56.3 
 31.7 
 25.1 
 74.8 
 40.8 
 31.7 
 87.0 
 55.7 
 49.5 
 80.8 
 52.1 
 45.0 

 15 
 80.1 
 59.6 
 51.7 
 57.3 
 31.1 
 24.3 
 77.7 
 42.6 
 33.2 
 88.2 
 55.9 
 49.5 
 82.7 
 52.7 
 45.4 

 20 
 79.8 
 59.4 
 51.1 
 58.3 
 30.3 
 24.4 
 81.2 
 45.3 
 35.5 
 88.5 
 55.6 
 49.1 
 83.4 
 52.9 
 45.5 

 30 
 80.4 
 60.0 
 52.0 
 60.4 
 29.9 
 23.0 
 79.4 
 44.3 
 35.0 
 88.4 
 56.1 
 49.5 
 83.3 
 53.1 
 45.7 

 Table 12: Category-wise breakdown of retrieval count k k on LoCoMo for gpt-4o-mini and gpt-4.1-mini. Semantic memory count is fixed at m = 2 ​ k m=2k . Bold rows mark the default setting ( k = 10 k{=}10 ) used in main experiments. 

 Index 
 Retrieve 
 Temporal 
 Open Domain 
 Multi-Hop 
 Single-Hop 
 Overall 

 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 
 LLM 
 F1 
 BLEU 

 N 
 N 
 72.9 
 55.7 
 48.3 
 55.2 
 26.0 
 20.4 
 72.3 
 41.8 
 32.8 
 82.3 
 53.3 
 47.1 
 76.9 
 50.0 
 42.9 

 P 
 N 
 71.7 
 56.5 
 49.0 
 47.9 
 24.0 
 19.6 
 72.3 
 40.5 
 30.8 
 82.8 
 55.0 
 48.7 
 76.4 
 50.7 
 43.7 

 N 
 P 
 73.8 
 43.4 
 35.8 
 55.2 
 27.3 
 21.5 
 72.0 
 43.2 
 32.4 
 82.4 
 57.8 
 51.0 
 77.0 
 50.2 
 42.6 

 P 
 P 
 69.5 
 42.5 
 35.2 
 51.0 
 22.6 
 18.2 
 67.0 
 40.2 
 30.5 
 83.0 
 59.4 
 52.4 
 75.3 
 50.1 
 42.7 

 Table 13: Category-wise breakdown of retrieval strategy ablation on LoCoMo using gpt-4.1-mini. N = narrative episodes; P = raw (partitioned) episodes. Index denotes the embedding source; Retrieve denotes the content returned to the LLM. When Retrieve = N, the top-2 narratives additionally include their raws, as described in Section  4.1 . Bold row marks the default setting. 

 Model 
 System 
 Input 
 Temp 
 Open 
 Multi 
 Single 
 Average 
 Core 

 gpt-4o-mini 
 A-MEM 
 N 
 54.2 
 22.9 
 43.6 
 58.2 
 52.5 
 52.6 

 𝒦 \mathcal{K} 
 33.6 
 38.5 
 50.4 
 59.1 
 50.9 
 55.8 

 Δ \Delta 

 ↓ \downarrow 38.0% 

 ↑ \uparrow 68.1% 

 ↑ \uparrow 15.4% 

 ↑ \uparrow 1.6% 

 ↓ \downarrow 3.0% 

 ↑ \uparrow 6.1% 

 MemoryOS 
 N 
 38.0 
 45.8 
 52.5 
 62.5 
 54.6 
 59.2 

 𝒦 \mathcal{K} 
 30.8 
 44.8 
 58.5 
 62.4 
 54.0 
 60.3 

 Δ \Delta 

 ↓ \downarrow 18.9% 

 ↓ \downarrow 2.2% 

 ↑ \uparrow 11.5% 

 ↓ \downarrow 0.2% 

 ↓ \downarrow 1.1% 

 ↑ \uparrow 1.9% 

 gpt-4.1-mini 
 A-MEM 
 N 
 66.7 
 37.5 
 55.7 
 64.0 
 61.4 
 60.4 

 𝒦 \mathcal{K} 
 41.1 
 41.7 
 58.2 
 68.0 
 59.0 
 64.1 

 Δ \Delta 

 ↓ \downarrow 38.4% 

 ↑ \uparrow 11.1% 

 ↑ \uparrow 4.5% 

 ↑ \uparrow 6.3% 

 ↓ \downarrow 3.9% 

 ↑ \uparrow 6.1% 

 MemoryOS 
 N 
 37.7 
 60.4 
 62.4 
 68.9 
 60.7 
 66.9 

 𝒦 \mathcal{K} 
 32.7 
 58.3 
 62.4 
 72.3 
 61.4 
 69.2 

 Δ \Delta 

 ↓ \downarrow 13.3% 

 ↓ \downarrow 3.5% 
 – 

 ↑ \uparrow 5.0% 

 ↑ \uparrow 1.2% 

 ↑ \uparrow 3.4% 

 Table 14: LLM Score comparison of third-party management systems with different input sources on LoCoMo. N=raw conversation; 𝒦 \mathcal{K} = Nemori ’s distilled semantic memory. Core=weighted average excluding Temporal. 

 Model 
 System 
 Input 
 Tokens 
 Chars 
 Entries 

 gpt-4o-mini 
 A-MEM 
 N 
 396,812 
 2,475,511 
 5,882 

 𝒦 \mathcal{K} 
 141,682 
 820,025 
 2,725 

 Δ \Delta 

 ↓ \downarrow 64.3% 

 ↓ \downarrow 66.9% 

 ↓ \downarrow 53.7% 

 MemoryOS 
 N 
 404,611 
 1,956,432 
 3,014 

 𝒦 \mathcal{K} 
 189,662 
 927,678 
 2,613 

 Δ \Delta 

 ↓ \downarrow 53.1% 

 ↓ \downarrow 52.6% 

 ↓ \downarrow 13.3% 

 gpt-4.1-mini 
 A-MEM 
 N 
 498,234 
 3,811,770 
 5,882 

 𝒦 \mathcal{K} 
 242,801 
 1,459,777 
 2,676 

 Δ \Delta 

 ↓ \downarrow 51.3% 

 ↓ \downarrow 61.7% 

 ↓ \downarrow 54.5% 

 MemoryOS 
 N 
 354,463 
 1,712,305 
 3,017 

 𝒦 \mathcal{K} 
 193,744 
 1,023,709 
 2,383 

 Δ \Delta 

 ↓ \downarrow 45.3% 

 ↓ \downarrow 40.2% 

 ↓ \downarrow 21.0% 

 Table 15: Memory storage comparison of third-party management systems with different input sources. N = raw conversation; 𝒦 \mathcal{K} = Nemori ’s distilled semantic memory. Entries denotes the number of memory entries defined by each system’s own storage format (comparable within but not across systems). Tokens and Chars are measured by concatenating all entries. ↓ \downarrow indicates reduction. 

Memory Construction Cost (Table  9 )

Finer-grained results of Section  4.3 . The main cost comes from the narrative episode generation (38.3%) and semantic knowledge distillation (30.3%).

Ablation Study (Table  10 ).

Finer-grained results of Section  4.4 .
Prediction-error-based distillation consistently outperforms direct knowledge distillation across categories.
The improvement is most pronounced in Temporal Reasoning , from 33.3 to 57.9 ( +73.9% ) on gpt-4o-mini and from 46.4 to 63.2 ( +36.2% ) on gpt-4.1-mini, where prediction-error-based distillation effectively identifies and transforms time-sensitive information.

Observation Window Length (Table  11 ).

Finer-grained results of Figure  2 .
Overall scores remain stable ( ± \pm 1%) across window lengths from 5 to 40.
Category-level variation is likewise small, confirming that Nemori ’s design of message partitioning with integration is robust to this hyperparameter.

Top-K Sensitivity (Table  12 ).

Performance rises sharply as k k increases from 2 to 10, then plateaus. Strongest average performance is achieved at k k =15 ∼ \sim 20, but k k =10 provides 97% of peak performance with lower computational cost.

Retrieval Strategy (Table  13 ).

The N → \rightarrow P configuration achieves a marginally higher LLM score (77.0 vs. 76.9), as raw text preserves factual details for answer generation.
We default to N → \rightarrow N for simplicity, since the difference is negligible and narrative retrieval avoids returning lengthy raw episodes.

Third-Party Management (Tables  14 and 15 ).

Using Nemori ’s semantic memory as input reduces storage by 45–64% while improving Core scores by 1.9–6.1%, demonstrating that distilled memory provides a compact yet information-rich representation suitable for downstream management systems.

 Appendix D Prompt Templates

This appendix provides the complete prompt templates used in Nemori ’s pipeline.

 D.1 Core Distillation Prompts

This subsection presents the prompts for the main distillation modules described in Section  3.2 and Section  3.3 , instantiated with our native management implementation.

 D.1.1 Local Message Partitioning Prompt ( 𝒫 par \mathcal{P}_{\text{par}} )

 Local Message Partitioning Prompt 

 You are an intelligent conversation segmentation expert. Your task is to analyze a batch of messages and group them into coherent episodes. 

 You will receive {count} messages numbered from 1 to {count}: {messages} 

 ## Your Task

Analyze these messages and group them into coherent episodes with **HIGH SENSITIVITY** to topic shifts. Be strict and create NEW episodes when detecting:

 1. **Topic Change** (Highest Priority): 
 - Do the new messages introduce a completely different topic? 
 - Is there a shift from one specific event to another? 
 - Has the conversation moved from one question to an unrelated new question? 
 2. **Intent Transition**: 
 - Has the purpose of the conversation changed? (e.g., from casual chat to seeking help, from discussing work to discussing personal life) 
 - Has the core question or issue of the current topic been answered or fully discussed? 
 3. **Temporal Markers**: 
 - Are there temporal transition markers ("earlier", "before", "by the way", "oh right", "also", etc.)? 
 - Is the time gap between messages more than 30 minutes? 
 4. **Structural Signals**: 
 - Are there explicit topic transition phrases ("changing topics", "speaking of which", "quick question", etc.)? 
 - Are there concluding statements indicating the current topic is finished? 
 5. **Content Relevance**: 
 - How related is the new message to the previous discussion? (Consider splitting if relevance < 30%) 
 - Does it involve completely different people, places, or events? 
 Decision Principles: 
 - **Prioritize topic independence**: Each episode should revolve around one core topic or event 
 - **When in doubt, split**: When uncertain, lean towards starting a new episode 
 - **Maintain reasonable length**: A single episode typically shouldn’t exceed 10-15 messages 
 ## Output Format 
 Return a JSON object with episodes, where each episode contains: 
 - ‘indices‘: List of message numbers (1-based) belonging to this episode 
 - ‘topic‘: Brief, specific description of what this episode is about 
 Example output: 
 {{ 
 "episodes": [ 
 {{ 
 "indices": [1, 2, 3, 4], 
 "topic": "Discussion about weekend hiking plans" 
 }}, 
 {{ 
 "indices": [5, 6, 7], 
 "topic": "Questions about Python programming" 
 }}, 
 {{ 
 "indices": [8, 9], 
 "topic": "Work schedule discussion" 
 }} 
 ] 
 }} 
 ## Important Guidelines 
 - Episodes can have non-consecutive indices if messages are interleaved 
 - An episode should typically contain 2-15 messages 
 - Focus on topical coherence over strict chronological order 
 - When in doubt, prefer smaller, more focused episodes 
 Return only the JSON object, no additional text. 

 D.1.2 Narrative Episode Generation Prompt ( 𝒫 nar \mathcal{P}_{\text{nar}} )

 Narrative Episode Generation Prompt 

 You are an episodic memory generation expert. Please convert the following conversation into an episodic memory. 

 Conversation content: {conversation} 

 Boundary detection reason: {boundary_reason} 

 Please analyze the conversation to extract time information and generate a structured episodic memory. Return only a JSON object containing the following three fields:

{{

"episodic_cue": "A concise, descriptive title that accurately summarizes the theme (10-20 words)",

"narrative_episode": "A detailed description of the conversation in third-person narrative. It must include all important information: who participated in the conversation at what time, what was discussed, what decisions were made, what emotions were expressed, and what plans or outcomes were formed. Write it as a coherent story so that the reader can clearly understand what happened. Ensure that time information is precise to the hour, including year, month, day, and hour.",

"timestamp": "YYYY-MM-DDTHH:MM:SS format timestamp representing when this episode occurred (analyze from message timestamps or content)"

}} 

 Time Analysis Instructions: 
 1. **Primary Source**: Look for explicit timestamps in the message metadata or content 
 2. **Secondary Source**: Analyze temporal references in the conversation content ("yesterday", "last week", "this morning", etc.) 
 3. **Fallback**: If no time information is available, use a reasonable estimate based on context 
 4. **Format**: Always return timestamp in ISO format: "2024-01-15T14:30:00" 
 Requirements: 
 1. The title should be specific and easy to search (including key topics/activities). 
 2. The content must include all important information from the conversation. 
 3. Convert the dialogue format into a narrative description. 
 4. Maintain chronological order and causal relationships. 
 5. Use third-person unless explicitly first-person. 
 6. Include specific details that aid keyword search. 
 7. Notice the time information, and write the time information in the content. 
 8. When relative times (e.g., last week, next month, etc.) are mentioned in the conversation, you need to convert them to absolute dates (year, month, day). Write the converted time in parentheses after the original time reference. 
 9. **IMPORTANT**: Analyze the actual time when the conversation happened from the message timestamps or content, not the current time. 
 Example: 
 If the conversation is about someone planning to go hiking and the messages have timestamps from March 14, 2024 at 3:00 PM:
{{ 
 "title": "Weekend Hiking Plan March 16, 2024: Sunrise Trip to Mount Rainier", 
 "content": "On March 14, 2024 at 3:00 PM, the user expressed interest in going hiking on the upcoming weekend (March 16, 2024) and sought advice. They particularly wanted to see the sunrise at Mount Rainier, having heard the scenery is beautiful. When asked about gear, they received suggestions including hiking boots, warm clothing (as it’s cold at the summit), a flashlight, water, and high-energy food. The user decided to leave at 4:00 AM on Saturday, March 16, 2024 to catch the sunrise and planned to invite friends for the adventure. They were very excited about the trip, hoping to connect with nature.", 
 "timestamp": "2024-03-14T15:00:00" 
 }} 
 Return only the JSON object, do not add any other text: 

 D.1.3 Optimal Candidate Identification Prompt ( 𝒫 sel \mathcal{P}_{\text{sel}} )

 Optimal Candidate Identification Prompt 

 You are an episodic memory merge decision expert. Determine if a new episode should be merged with an existing similar episode. 
 ## New Episode 

 Time Range: {new_time_range} 

 Content: {new_content} 

 Candidate Episodes to Merge With: {candidates} 

 ## Your Task

Decide whether the new episode should:

1. **merge**: Merge with one of the candidates (they describe the same event/topic)

2. **new**: Keep as a separate new episode (it’s a distinct event)

 ## Merge Criteria 
 Merge ONLY if: 
 - Both episodes describe the SAME event or conversation session 
 - They have significant temporal overlap or are very close in time 
 - The content is clearly a continuation or different perspective of the same topic 
 - Merging would create a more comple