---
title: "What Deserves Memory: Adaptive Memory Distillation for LLM Agents"
authors: ["Wenquan Ma", "Jiayan Nan", "Wenlong Wu", "Yize Chen", "Corresponding author."]
url: "https://arxiv.org/abs/2508.03341"
sections: 79
estimated_tokens: "24.6k"
---

## Contents
- 1 Introduction
- 2 Related Works
  - 2.1 Memory for LLM Agents
  - 2.2 Cognitive Principles of Memory
- 3 Methodology
  - 3.1 Overview & Motivations
    - The Structure Prior: Integrity of Episode.
    - The Representation Prior: Asymmetry of Perspective.
    - The Distillation Prior: Predictability Implies Redundancy.
  - 3.2 Episodic Memory Integration
    - 3.2.1 Local Message Partitioning
    - 3.2.2 Narrative Episode Generation
      - Discussion.
    - 3.2.3 Associative Memory Integration
  - 3.3 Semantic Knowledge Distillation
    - 3.3.1 Anticipatory Schema Synthesis
    - 3.3.2 Prediction Error Distillation
    - 3.3.3 Agnostic Knowledge Consolidation
  - 3.4 Response Generation
- 4 Experiments
  - 4.1 Experimental Setup
    - Datasets.
    - Baselines.
    - Evaluation Metrics.
    - Implementation Details.
  - 4.2 Main Results (RQ1)
    - Strong Performance.
    - Exceptional Temporal Reasoning.
  - 4.3 Efficiency Analysis (RQ2)
    - Memory Construction.
    - Response Generation.
  - 4.4 Ablation Study (RQ3)
    - Prediction-error-based vs. Direct Distillation.
    - Native Management Contribution.
    - Episodic–Semantic Complementarity.
    - Observation Window Length.
  - 4.5 Retrieval Hyperparameter Analysis (RQ4)
    - Top-K Sensitivity.
    - Index–Retrieve Strategy.
  - 4.6 Third-Party Integration (RQ5)
  - 4.7 Scalability Analysis (RQ6)
- 5 Conclusion
- Limitations
- Acknowledgments
- References
- Appendix A Implementations of Management
  - A.1 Conceptual Model: Flat Summarization
    - Context Evocation.
    - Knowledge Consolidation.
  - A.2 Variant: Naive RAG
    - Context Evocation.
    - Knowledge Consolidation.
  - A.3 External Integration: Third-Party Systems
    - Context Evocation.
    - Knowledge Consolidation.
- Appendix B Case Study
  - B.1 Temporal Reasoning
  - B.2 Open Domain
- Appendix C Additional Experiment Results
  - Memory Construction Cost (Table 9 )
  - Ablation Study (Table 10 ).
  - Observation Window Length (Table 11 ).
  - Top-K Sensitivity (Table 12 ).
  - Retrieval Strategy (Table 13 ).
  - Third-Party Management (Tables 14 and 15 ).
- Appendix D Prompt Templates
  - D.1 Core Distillation Prompts
    - D.1.1 Local Message Partitioning Prompt ( 𝒫 par \mathcal{P}_{\text{par}} )
    - D.1.2 Narrative Episode Generation Prompt ( 𝒫 nar \mathcal{P}_{\text{nar}} )
    - D.1.3 Optimal Candidate Identification Prompt ( 𝒫 sel \mathcal{P}_{\text{sel}} )
    - D.1.4 Episodic Integration Prompt ( 𝒫 int \mathcal{P}_{\text{int}} )
    - D.1.5 Anticipatory Schema Synthesis Prompt ( 𝒫 ant \mathcal{P}_{\text{ant}} )
    - D.1.6 Prediction Error Distillation Prompt ( 𝒫 dis \mathcal{P}_{\text{dis}} )
    - D.1.7 Semantic Consolidation Prompt ( 𝒫 con \mathcal{P}_{\text{con}} )
  - D.2 Direct Distillation Prompt ( Nemori -s)
  - D.3 Response Generation Prompt ( 𝒫 ans \mathcal{P}_{\text{ans}} )
  - D.4 LLM-as-Judge Prompts
    - D.4.1 LoCoMo
    - D.4.2 LongMemEval S {}_{\text{S}}

## Abstract

Abstract Memory systems for LLM agents struggle to determine what information deserves retention.
Existing approaches rely on predefined heuristics such as importance scores, emotional tags, or factual templates, encoding designer intuition rather than learning from the data itself.
Inspired by cognitive ideas, we propose Nemori , an adaptive memory distillation framework that casts the assessment of the experience’s future utility as a matter of predictability.
Specifically, Nemori comprises two cascading modules: Episodic Memory Integration transforms raw interactions into coherent narratives, and Semantic Knowledge Distillation extracts insights via prediction error.
Centering on distillation, the framework remains agnostic to downstream management.
Extensive experiments confirm that Nemori achieves strong performance, efficiency, and storage reduction.
Our work suggests that observing the intrinsic properties of interaction sequences offers a viable, data-driven alternative to heuristic-based memory design.
Code at: https://github.com/nemori-ai/nemori .

## 1 Introduction

The difficulty of maintaining long-term behavioral consistency in Large Language Model (LLM)-based agents stems from a fundamental conflict: the reliance of stateless LLMs on linearly expanding interaction trajectories versus the constraints of finite context window and the *Lost in the Middle* phenomenon (Liu et al., 2024).
Nevertheless, real-world applications, exemplified by personal assistants, autonomous agents and personalized recommendation systems, increasingly demand persistent interaction.
To address these challenges, memory systems that facilitate real-time context regulation have emerged as a viable and prevailing approach.

**Table 1: Agent memory systems categorized by the stage at which memory utility is assessed. Retrieval-time methods defer assessment entirely; management-time methods filter post-hoc via access patterns; distillation-time methods assess at entry ingestion. Common practices omitted: raw retention in Distillation, conflict detection in Management, similarity search (non-graph) or graph traversal (graph-based) in Retrieval. Our approach assesses at distillation via prediction error rather than predefined heuristics.**
| Category | Method | Distillation | Management | Retrieval |
| --- | --- | --- | --- | --- |
| Retrieval-time | Lewis et al. (2020) | — | — | Similarity search |
| Management-time | Packer et al. (2023) | — | Tiered storage | Function calls |
| Zhong et al. (2024) | — | Summary + forgetting | — |  |
| Kang et al. (2025) | — | Heat scoring | Two-step search |  |
| Li et al. (2025b) | — | Hierarchical summary | Multi-step search |  |
| Anokhin et al. (2025) | — | Graph update | Graph spreading |  |
| Xu et al. (2025) | — | Adaptive note linking | — |  |
| Rasmussen et al. (2025) | — | Validity management | Reranking |  |
| Distillation-time | Park et al. (2023) | Importance scoring | Reflection trees | Weighted scoring |
| Huang et al. (2024) | Emotion tagging | — | Emotion matching |  |
| Chhikara et al. (2025) | Facts extraction | — | — |  |
| Li et al. (2025a) | Summary + persona | — | Noun overlap |  |
| Pan et al. (2025) | Topic | Token compression | — |  |
|  | This paper | Prediction error | Agnostic | — |

Memory systems identify useful experiences to facilitate future response generation through two stages: *distillation*, which determines the entry form of experiences, and *management*, which ensures their ongoing maintenance.
To this end, one category of approaches focuses on management by treating entries as opaque containers, where utility is inferred through observable structural metadata, such as access frequency (Kang et al., 2025), temporal decay (Zhong et al., 2024), or explicit relationships (Xu et al., 2025), foregoing the inspection of the nuanced content itself.
In contrast, another category of approaches intervenes during the initial distillation stage, selectively shaping the entry form.
This *pre-positioning*, while granting greater flexibility, must contend with future utility uncertainty.
Existing distillation methods typically address this by encoding designer intuition, such as importance scores (Park et al., 2023), emotional tags (Huang et al., 2024), or factual templates (Chhikara et al., 2025).
However, such heuristics risk introducing subjective bias, which is fatal during distillation as it can lead to irreversible information distortion, or causing systemic bloat, where the system tends to over-store to avoid such distortion, thereby amplifying retrieval noise.
This limitation necessitates an approach that assesses the potential utility grounded in the interaction experience itself.

Inspired by Predictive Coding Theory (Rao and Ballard, 1999; Friston, 2010; Clark, 2013), we propose Nemori, a training-free framework that casts the assessment of experience utility as adaptive memory distillation over incoming observations that the agent fails to predict given existing knowledge, enabling a data-driven space.
As illustrated in Figure [1](#S2.F1), this framework, guided by three parsimonious priors over memory structure, representation and distillation, comprises two cascading modules, echoing Complementary Learning Systems (McClelland et al., 1995).
Specifically, the *Episodic Memory Integration* module first transforms raw interaction sequences into coherent episodic narratives.
The *Semantic Knowledge Distillation* module then extracts novel experience that existing knowledge cannot anticipate.
Centering on the distillation stage, Nemori remains agnostic to the underlying management, while a native management system is provided.
Our contributions:

1) Perspective. We formalize the distinction between *distillation* and *management* in memory construction, and derive priors from general data properties and cognitive ideas to guide distillation design.

2) Framework. We implement Nemori, a management-agnostic adaptive memory distillation framework, and equip it with a native management system.

3) Evaluation. We conduct extensive experiments demonstrating Nemori’s strong performance, with pronounced advantages in longer context. When integrated with third-party management systems, Nemori enhances A-MEM and MemoryOS with 45–64% storage reduction while maintaining performance.

## 2 Related Works

### 2.1 Memory for LLM Agents

Agent memory systems decompose into three stages: distillation (what to retain), management (how to organize), and retrieval (how to surface content).
Unlike pure RAG (Lewis et al., 2020) that defers judgment to query time, memory systems *pre-position*: enriching data with metadata at distillation, then utilizing it for management and retrieval.
Table [1](#S1.T1) categorizes works by when enrichment occurs.
*Management-time* methods enrich post-hoc via decay weights, tiered storage, access frequency, or relationship linking (Zhong et al., 2024; Packer et al., 2023; Kang et al., 2025; Xu et al., 2025).
*Distillation-time* methods enrich at ingestion through importance scoring, emotional tagging, or fact extraction (Park et al., 2023; Huang et al., 2024; Chhikara et al., 2025).

Figure: Figure 1: Overview of the Nemori framework. The system comprises two cascading modules guided by three priors: Episodic Memory Integration (top) transforms raw interactions into coherent narrative episodes, and Semantic Knowledge Distillation (bottom) extracts insights via prediction error. The framework can serve as a distillation layer complement native or third-party management systems (right).
Refer to caption: 2508.03341v4/x1.png

### 2.2 Cognitive Principles of Memory

Predictive Coding Theory (Rao and Ballard, 1999), originally from visual neuroscience, posits that higher cortical areas send predictions downward while lower areas propagate primarily the residual *prediction error* upward.
Friston (2010) generalized this into the Free Energy Principle, a unifying framework across perception, action, and learning.
Clark (2013) further extended it, arguing that brains are fundamentally prediction machines.
Nemori adapts this insight to agent memory design: prediction error signals information worth retaining; what is predictable is therefore redundant.

## 3 Methodology

Nemori is an adaptive memory distillation framework inspired by cognitive ideas (McClelland et al., 1995; Rao and Ballard, 1999; Friston, 2010; Clark, 2013).
This management-agnostic framework can serve as a distillation layer that complements either native or third-party memory systems like A-MEM or MemoryOS.
A production-grade implementation is provided at [https://github.com/nemori-ai/nemori](https://github.com/nemori-ai/nemori).

### 3.1 Overview & Motivations

As illustrated in Figure [1](#S2.F1), Nemori comprises two cascading modules guided by three priors as inductive biases. These priors capture the parsimonious features of continuous interaction sequences, establishing a plastic, data-driven environment in which intrinsic dynamics drive the partitioning, representation, and distillation of experience into memory.

##### The Structure Prior: Integrity of Episode.

Interaction sequences exhibit natural grouping.
Interactions within each episodic group are mutually contextualizing: individual messages derive their meaning, partly, from surrounding ones, and finer-grained or arbitrary fragmentation would sever the context that renders them interpretable.

This prior requires the framework to define episodes respecting latent integrity among interactions, rather than imposing heuristic chunking.

##### The Representation Prior: Asymmetry of Perspective.

Memory serves recall.
Recalling is essentially a form of reasoning, an allocentric reconstruction of events, whereas raw episodes are egocentric and inherently noisy.

This prior requires the framework to transform raw episodes into narrative representations that highlight logical structures while preserving salient details, bridging the gap between chaotic perception and rational retrieval.

##### The Distillation Prior: Predictability Implies Redundancy.

Information within interaction sequences is highly redundant.
From the perspective of predictive coding, the unexpected information is a natural candidate for memory consolidation.

This key prior requires the framework to distill memory by inspecting the semantic differential between the actual interactions and their anticipatory schema derived from existing knowledge.

In the following sections, we detail the framework implementation guided by these priors.

### 3.2 Episodic Memory Integration

Guided by the structure prior and the representation prior, this module integrates raw interactions into episodic memories and prepares them for subsequent distillation.
It is further divided into three submodules: Local Message Partitioning, Narrative Episode Generation and Associative Memory Integration.

#### 3.2.1 Local Message Partitioning

Guided by the structure prior, this submodule resolves the continuous interaction sequence within an observation window into a discrete partition.
We model the interactions between an agent and its environment as a sequence of message exchanges, maintaining a dedicated message buffer $\mathcal{B}$.
At any time $t$, the buffer state is represented as a queue of messages $\mathcal{B}_{t}=\{m_{1},m_{2},\dots,m_{z}\}$, where each message $m_{i}=(r_{i},c_{i},\tau_{i})$ specifies the sender, content and timestamp, respectively.
New interactions are appended to the rear of $\mathcal{B}_{t}$ as they occur.

The partitioning process is triggered once the buffer size $|\mathcal{B}_{t}|$ reaches a predefined observation window length $w\in\mathbb{Z}^{+}$.
At this juncture, the submodule performs a partitioning operation:

$$ $\mathbf{O}\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{par}}\parallel\mathcal{B}_{t}),$ $$

where $\mathcal{P}_{\text{par}}$ is a prompt that instructs the LLM to discern the latent integrity and local nuances within the window and partition the messages accordingly.
The output $\mathbf{O}=\{O_{1},O_{2},\dots,O_{n}\}$ (where $n\leq w$) constitutes a partition of the index set $\{1,2,\dots,w\}$.
Specifically, the $O_{j}$ are pairwise disjoint and their union covers the index set.
The submodule then maps these indices back to the message buffer to form a collection of raw episodes $\mathbf{P}=\{P_{1},\dots,P_{n}\}$, where each $P_{j}$ is the subsequence of $\mathcal{B}_{t}$ indexed by $O_{j}$.

Finally, $\mathbf{P}$ is transferred to the Narrative Episode Generation submodule, and the buffer $\mathcal{B}$ is reset to empty to await subsequent incoming messages.

#### 3.2.2 Narrative Episode Generation

Guided by the representation prior, this submodule transforms the received raw episodes into narrative representations.
For each raw episode $P_{j}\in\mathbf{P}$, the submodule generates a narrative episode $N_{j}$ and a corresponding episodic cue $c_{j}$ tailored for semantic distillation:

$$ $(N_{j},c_{j})\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{nar}}\parallel P_{j}),$ $$

where $\mathcal{P}_{\text{nar}}$ instructs the LLM to highlight the logical structure and constituent elements within the interaction.

Subsequently, the submodule computes an embedding index $\mathbf{v}_{j}$ to enable associative retrieval:

$$ $\mathbf{v}_{j}\leftarrow f_{\text{emb}}(c_{j}\parallel N_{j}),$ $$

where $f_{\text{emb}}$ denotes the embedding model.

Each episodic memory is represented as $M_{j}=(c_{j},N_{j},P_{j},\mathbf{v}_{j})$. Finally, the collection $\{M_{j}\}$ is transferred to the Associative Memory Integration submodule.

##### Discussion.

Two aspects of this design merit attention. First, it enables dual-mode retrieval: returning $N$ directly for efficiency, or returning raw $P$ for precision-critical domains. Second, from this point onward, the episode becomes the primary processing unit throughout the pipeline, avoiding the *message-wise* processing that many baselines fall into and that incurs substantial cost overhead (see Section [4.3](#S4.SS3)).

#### 3.2.3 Associative Memory Integration

This submodule dynamically integrates episodes that may have been sundered by the constraints of the observation window length.
For each new episodic memory $M_{j}$, the submodule performs an integration check against the existing episodic database $\mathcal{D}_{e}$.
It first retrieves the top $K_{e}$ candidates based on cosine similarity:

$$ $\mathbf{C}=\{U_{1},U_{2},\dots,U_{K_{e}}\}\leftarrow\operatorname{Search}(\mathcal{D}_{e},\mathbf{v}_{j},K_{e}),$ $$

where each candidate $U_{k}=(c_{k},N_{k},P_{k},\mathbf{v}_{k})$.

The submodule then selects the optimal integration target:

$$ $idx\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{sel}}\parallel(c_{j},N_{j})\parallel\{(c_{k},N_{k})\}_{k=1}^{K_{e}}),$ $$

where $\mathcal{P}_{\text{sel}}$ instructs the LLM to identify the target candidate that shares episodic continuity with the new memory.
The output $idx\in\{1,\dots,K_{e}\}\cup\{-1\}$ determines the subsequent operation:

Case 1 ($idx=k$): The LLM integrates two memories: $(c_{\nu},N_{\nu})\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{int}}\parallel c_{k}\parallel N_{k}\parallel c_{j}\parallel N_{j})$, superseding $U_{k}$ with $M_{\nu}=(c_{\nu},N_{\nu},P_{k}\parallel P_{j},f_{\text{emb}}(c_{\nu}\parallel N_{\nu}))$.

Case 2 ($idx=-1$): No continuity found; $M_{j}$ is inserted as a distinct entry.

Finally, the resulting episodic memory ($M_{\nu}$ or $M_{j}$) is transferred to the Semantic Knowledge Distillation module.

### 3.3 Semantic Knowledge Distillation

Guided by the distillation prior, this module implements a management-agnostic process to distill semantic knowledge from episodic experiences by defining generic interfaces for context evocation and knowledge consolidation.
This process comprises three submodules: Anticipatory Schema Synthesis, Prediction Error Distillation, Agnostic Knowledge Consolidation.

#### 3.3.1 Anticipatory Schema Synthesis

This submodule synthesizes an anticipatory schema for each incoming episode by orchestrating existing knowledge.
We treat the underlying management system $\mathcal{M}$ as an abstract context provider.
Let $M_{in}$ denote the input episodic memory, newly formed or integrated alike.
The submodule first invokes a generic interface $\operatorname{Evoke}(\cdot)$ to evoke the context $\mathcal{S}_{in}$ pertaining to $M_{in}$ from $\mathcal{M}$:

$$ $\mathcal{S}_{in}\leftarrow\operatorname{Evoke}(M_{in},\mathcal{M}).$ $$

In our native implementation, this is realized as threshold-filtered similarity search: $\mathcal{S}_{in}\leftarrow\operatorname{Top-}K_{s}\left({S_{r}\in\mathcal{D}_{s}\mid\operatorname{sim}(\mathbf{v}_{in},\mathbf{u}_{r})>\tau}\right)$.
Alternative variants are detailed in Appendix [A](#A1).

The anticipatory schema $\hat{P}_{in}$ is then synthesized given only the episodic cue $c_{in}$ (a brief summary of the incoming episode) and the evoked context $\mathcal{S}_{in}$ (what the system already knows):

$$ $\hat{P}_{in}\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{ant}}\parallel c_{in}\parallel\mathcal{S}_{in}),$ $$

where $\mathcal{P}_{\text{ant}}$ instructs the LLM to *predict what actually happened* in the incoming episode based on the provided context.
The resulting $\hat{P}_{in}$ represents the system’s guess of the episode content from existing knowledge alone.

#### 3.3.2 Prediction Error Distillation

This submodule distills semantic insights from the discrepancy between the raw episode and the anticipatory schema. The process is defined as:

$$ $\mathcal{K}_{in}=\{k_{1},\dots,k_{d}\}\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{dis}}\parallel P_{in}\parallel\hat{P}_{in}),$ $$

where $\mathcal{P}_{\text{dis}}$ instructs the LLM to identify and extract, as semantic insights $\mathcal{K}_{in}$, information in the raw episode $P_{in}$ that deviates from or extends the anticipatory schema $\hat{P}_{in}$.

#### 3.3.3 Agnostic Knowledge Consolidation

This submodule consolidates the distilled $\mathcal{K}_{in}$ into the underlying management system $\mathcal{M}$, defined by a generic interface $\operatorname{Consolidate}(\mathcal{K}_{in},\mathcal{M})$, with diverse implementations detailed in Appendix [A](#A1).

In our native implementation, for each distilled insight $k_{q}\in\mathcal{K}_{in}$, the submodule first retrieves its associative knowledge $\tilde{\mathcal{S}}_{q}$ from the semantic database $\mathcal{D}_{s}$ based on the embedding $\mathbf{u}_{q}\leftarrow f_{\text{emb}}(k_{q})$:

$$ $\tilde{\mathcal{S}}_{q}=\{(k_{h},\mathbf{u}_{h})\}_{h=1}^{K_{m}}\leftarrow\operatorname{Search}(\mathcal{D}_{s},\mathbf{u}_{q},K_{m}).$ $$

The submodule then resolves the relationship between the insight and existing knowledge, generating a consolidation directive:

$$ $(\delta,idxs,k_{\mu})\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{con}}\parallel k_{q}\parallel\{k_{h}\}_{h=1}^{K_{m}}),$ $$

where $\mathcal{P}_{\text{con}}$ instructs the LLM to determine consolidation operations, $\delta\in\{\text{{new}},\text{{merge}},\text{{conflict}}\}$ specifies the consolidation strategy, $idxs\subseteq\{1,\dots,K_{m}\}$ are the indices of target entries in $\tilde{\mathcal{S}}_{q}$, and $k_{\mu}$ is the potential consolidated content.

The final consolidation operation follows three branching cases: new inserts $(k_{q},\mathbf{u}_{q})$ as a distinct entry when no overlap is detected; merge supersedes entries indexed by $idxs$ with unified $(k_{\mu},f_{\text{emb}}(k_{\mu}))$ when the insight complements existing knowledge; conflict purges outdated entries and replaces with $(k_{q},\mathbf{u}_{q})$ when $k_{q}$ invalidates previous knowledge.

Algorithm [1](#alg1) in the Appendix summarizes Nemori’s memory construction pipeline.

### 3.4 Response Generation

The inference-time use of memory is largely orthogonal to its construction procedures in Section [3.2](#S3.SS2) and Section [3.3](#S3.SS3). Therefore, response generation can in principle accommodate diverse retrieval strategies; here we present the direct setting used in our experiments.

Given a user query $Q$ and its embedding $\mathbf{v}_{Q}\leftarrow f_{\text{emb}}(Q)$, we retrieve in parallel the top-$k$ episodic entries from the episodic database $\mathcal{D}_{e}$:

$$ $\tilde{\mathcal{R}}_{e}=\{(c_{i},N_{i},P_{i},\mathbf{v}_{i})\}_{i=1}^{k}\leftarrow\operatorname{Search}(\mathcal{D}_{e},\mathbf{v}_{Q},k),$ $$

and the top-$m$ semantic entries from the semantic database $\mathcal{D}_{s}$:

$$ $\tilde{\mathcal{R}}_{s}=\{(s_{j},\mathbf{u}_{j})\}_{j=1}^{m}\leftarrow\operatorname{Search}(\mathcal{D}_{s},\mathbf{v}_{Q},m).$ $$

Both $\tilde{\mathcal{R}}_{e}$ and $\tilde{\mathcal{R}}_{s}$ are ordered by decreasing similarity to the query embedding $\mathbf{v}_{Q}$.

The final context for response generation is the concatenation of narrative episodes $\mathcal{R}_{e}=\{N_{i}\}_{i=1}^{k}$, raw episodes $\mathcal{R}_{p}=\{P_{d}\}_{d=1}^{r}$, and semantic knowledge $\mathcal{R}_{s}=\{s_{j}\}_{j=1}^{m}$:

$$ $a\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{ans}}\parallel Q\parallel\mathcal{R}_{e}\parallel\mathcal{R}_{p}\parallel\mathcal{R}_{s}),$ $$

where $\mathcal{P}_{\text{ans}}$ instructs the LLM to generate a response to the question grounded in the retrieved context.

Algorithm [2](#alg2) in the Appendix summarizes the response generation procedure described above.

**Table 2: Performance comparison on LoCoMo dataset. Underline: strongest memory system (excluding Nemori). Improv.: Nemori’s relative improvement (%) over strongest memory system.**
|  | Method | Temporal Reasoning | Open Domain | Multi-Hop | Single-Hop | Average |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | $\uparrow$LLM | $\uparrow$F1 | $\uparrow$BLEU | $\uparrow$LLM | $\uparrow$F1 | $\uparrow$BLEU | $\uparrow$LLM | $\uparrow$F1 | $\uparrow$BLEU | $\uparrow$LLM | $\uparrow$F1 | $\uparrow$BLEU | $\uparrow$LLM | $\uparrow$F1 | $\uparrow$BLEU |  |
| gpt-4.1-mini | Full Context | 74.2 | 47.5 | 40.0 | 56.6 | 28.4 | 22.2 | 77.2 | 44.2 | 33.7 | 86.9 | 61.4 | 53.4 | 80.6 | 53.3 | 45.0 |
| RAG-4096 | 27.4 | 22.3 | 19.1 | 28.8 | 17.9 | 13.9 | 31.7 | 20.1 | 12.8 | 35.9 | 25.8 | 22.0 | 32.9 | 23.5 | 19.2 |  |
| LangMem | 50.8 | 48.5 | 40.9 | 59.0 | 32.8 | 26.4 | 71.0 | 41.5 | 32.5 | 84.5 | 51.0 | 43.6 | 73.4 | 47.6 | 40.0 |  |
| Zep | 60.2 | 23.9 | 20.0 | 43.8 | 24.2 | 19.3 | 53.7 | 30.5 | 20.4 | 66.9 | 45.5 | 40.0 | 61.6 | 36.9 | 30.9 |  |
| Mem0 | 56.9 | 39.2 | 33.2 | 47.9 | 23.7 | 17.7 | 68.2 | 40.1 | 30.3 | 71.4 | 48.6 | 42.0 | 66.3 | 43.5 | 36.5 |  |
| A-MEM | 66.7 | 40.3 | 33.7 | 37.5 | 13.4 | 12.7 | 55.7 | 30.4 | 20.0 | 64.0 | 45.0 | 39.8 | 61.4 | 39.4 | 33.2 |  |
| MemoryOS | 37.7 | 36.5 | 27.4 | 60.4 | 30.2 | 25.6 | 62.4 | 34.0 | 25.8 | 68.9 | 44.2 | 37.5 | 60.6 | 39.9 | 32.5 |  |
| Nemori | 77.3 | 58.7 | 50.7 | 56.3 | 31.7 | 25.1 | 74.8 | 40.8 | 31.7 | 87.0 | 55.7 | 49.5 | 80.8 | 52.1 | 45.0 |  |
| Improv. | $\uparrow$15.9% | $\uparrow$21.0% | $\uparrow$24.0% | – | – | – | $\uparrow$5.4% | – | – | $\uparrow$3.0% | $\uparrow$9.2% | $\uparrow$13.5% | $\uparrow$10.1% | $\uparrow$9.5% | $\uparrow$12.5% |  |
| gpt-4o-mini | Full Context | 56.2 | 44.1 | 36.1 | 48.6 | 24.5 | 17.2 | 66.8 | 35.4 | 26.1 | 83.0 | 53.1 | 44.7 | 72.3 | 46.2 | 37.8 |
| RAG-4096 | 23.7 | 19.5 | 15.7 | 32.6 | 19.0 | 13.5 | 31.3 | 18.6 | 11.7 | 32.0 | 22.2 | 18.6 | 30.2 | 20.8 | 16.4 |  |
| LangMem | 24.9 | 31.9 | 26.2 | 47.6 | 29.4 | 23.5 | 52.4 | 33.5 | 23.9 | 61.4 | 38.8 | 33.1 | 51.3 | 35.8 | 29.4 |  |
| Zep | 58.9 | 44.8 | 38.1 | 39.6 | 22.9 | 15.7 | 50.5 | 27.5 | 19.3 | 63.2 | 39.7 | 33.7 | 58.5 | 37.5 | 30.9 |  |
| Mem0 | 50.4 | 44.4 | 37.6 | 40.6 | 27.1 | 19.4 | 60.3 | 34.3 | 25.2 | 68.1 | 44.4 | 37.7 | 61.3 | 41.5 | 34.2 |  |
| A-MEM | 54.2 | 38.1 | 33.8 | 22.9 | 9.0 | 8.6 | 43.6 | 24.0 | 18.8 | 58.2 | 35.6 | 29.2 | 52.5 | 32.4 | 27.0 |  |
| MemoryOS | 38.0 | 38.5 | 27.5 | 45.8 | 26.0 | 19.2 | 52.5 | 35.2 | 24.1 | 62.5 | 43.7 | 37.7 | 54.5 | 39.9 | 31.9 |  |
| Nemori | 67.6 | 57.3 | 47.6 | 45.8 | 23.9 | 18.5 | 61.7 | 38.1 | 26.0 | 81.9 | 54.8 | 43.8 | 73.0 | 50.3 | 39.7 |  |
| Improv. | $\uparrow$14.8% | $\uparrow$27.9% | $\uparrow$24.9% | – | – | – | $\uparrow$2.3% | $\uparrow$8.2% | $\uparrow$3.2% | $\uparrow$20.3% | $\uparrow$23.4% | $\uparrow$16.2% | $\uparrow$19.1% | $\uparrow$21.2% | $\uparrow$16.1% |  |

## 4 Experiments

We conduct experiments addressing: (RQ1) performance comparison, (RQ2) efficiency analysis, (RQ3) component sensitivity, (RQ4) retrieval configurations, (RQ5) third-party management integration, and (RQ6) scalability to longer contexts.

### 4.1 Experimental Setup

##### Datasets.

We evaluate Nemori on two distinct benchmarks.
LoCoMo (Maharana et al., 2024): 10 dialogues with 24K average tokens, featuring 1,540 questions across four reasoning categories.
LongMemEval${}_{\text{S}}$ (Wu et al., 2025): 500 conversations with 105K average tokens. While structurally similar to LoCoMo, it presents significantly greater challenges through longer, more realistic conversational contexts, allowing us to assess scalability under demanding conditions.

##### Baselines.

We benchmark against seven methods:
Full Context (entire dialogue history),
RAG-4096 (Lewis et al., 2020, 4096-token chunks for dense retrieval),
LangMem (Chase, 2022, automatic knowledge extraction across sessions),
Zep (Rasmussen et al., 2025, fact extraction into knowledge graphs),
Mem0 (Chhikara et al., 2025, vector-based preference capture),
A-MEM (Xu et al., 2025, structured notes with evolving links), and
MemoryOS (Kang et al., 2025, hierarchical OS-inspired storage).

##### Evaluation Metrics.

On the LoCoMo dataset, our primary evaluation metric is the LLM-judge score (abbreviated as LLM for simplicity), using gpt-4o-mini as the judge. We additionally report F1 and BLEU-1. For the LongMemEval${}_{\text{S}}$ dataset, we also use the LLM-judge score, but with prompts adapted to its task-specific question-answering format, following Zep (Rasmussen et al., 2025). These metrics are accuracy metrics of different standard, and are scaled to the 0–100 range, with higher values indicating better performance and 100 denoting a perfect score.

##### Implementation Details.

To ensure fair comparison, Mem0 and Zep utilize their commercial APIs to retrieve memory contexts, which are then fed to gpt-4o-mini and gpt-4.1-mini for answer generation. All other methods, including Nemori, employ gpt-4o-mini and gpt-4.1-mini as both internal backbone models and answer generation models. For Nemori specifically, embeddings are generated with text-embedding-3-small. Key hyperparameters were configured as follows: similarity threshold $\tau=0.70$, distillation parameters $K_{e}=K_{m}=5,K_{s}=10$. For retrieval count, we fix $m=2k$; in main experiments $k=10$ (thus $m=20$), while $k$ varies from 2 to 30 in RQ3. To balance informativeness and efficiency, only the top-2 episodic memories include their original conversation text (i.e, $r=2$), as higher-similarity episodes tend to be more informative.

### 4.2 Main Results (RQ1)

Table [2](#S3.T2) presents the main performance comparison on LoCoMo.
Regarding this table, we highlight the following observations:

##### Strong Performance.

Nemori achieves the strongest average performance.
In terms of the average LLM-judge score, with gpt-4.1-mini, Nemori achieves 80.8, surpassing LangMem (73.4) by 10.1%; with gpt-4o-mini, it reaches 73.0, exceeding Mem0 (61.3) by 19.1%.
Importantly, Nemori slightly exceeds Full Context on both models (80.8 vs. 80.6 and 73.0 vs. 72.3), suggesting that it effectively captures the intrinsic properties of interaction sequences and recognizes useful experience.

##### Exceptional Temporal Reasoning.

Nemori excels in Temporal Reasoning, achieving LLM-judge score of 77.3 with gpt-4.1-mini (+15.9% over A-MEM) and 67.6 with gpt-4o-mini (+14.8% over Zep).
This result suggests the effectiveness of Nemori’s *episode-centric* design (§[3.2](#S3.SS2), §[3.3](#S3.SS3)), which front-loads part of the reasoning burden from response generation to memory formation and better aligns with the inherent logical structure of experience.
A case study is provided in Appendix [B.1](#A2.SS1).

Nemori’s performance is slightly lower compared to the strongest method on Open Domain. Questions in this category usually require both memory and the backbone model’s prior knowledge. See Appendix [B.2](#A2.SS2) for further discussion.

### 4.3 Efficiency Analysis (RQ2)

The efficiency analysis of the memory system can be divided into two stages: memory construction and response generation, where the former corresponds to the distillation and management stages and the latter corresponds to the retrieval stage in the sense of Section [1](#S1) and Table [1](#S1.T1).

##### Memory Construction.

**Table 3: Comparison of memory construction cost on LoCoMo using gpt-4o-mini. The last three columns report token consumption.**
| Method | $\uparrow$ LLM | $\downarrow$ Calls | $\downarrow$ Input | $\downarrow$ Output | $\downarrow$ Total |
| --- | --- | --- | --- | --- | --- |
|  | (%) |  | (k) | (k) | (k) |
| LangMem | 51.3 | 920.6 | 898.3 | 112.0 | 1010.2 |
| Mem0 | 61.3 | 1602.2 | 1483.4 | 210.0 | 1693.4 |
| A-MEM | 52.5 | 1175.5 | 912.6 | 236.8 | 1149.4 |
| MemoryOS | 54.5 | 1016.1 | 404.5 | 122.0 | 526.5 |
| Nemori | 73.0 | 373.2 | 277.2 | 45.7 | 322.9 |
| Improv. | $\uparrow$19.1% | $\downarrow$59.5% | $\downarrow$31.5% | $\downarrow$59.2% | $\downarrow$38.7% |

Table [3](#S4.T3) reports the cost of memory construction on LoCoMo with gpt-4o-mini, where the baseline results are taken from Fang et al. (2025) and Nemori is evaluated under the same scope for fair comparison.
Nemori is more cost-efficient than the baselines, reducing LLM calls by 59.5% and token consumption by 38.7%. This result may seem surprising, as Nemori appears to employ a complex pipeline with multiple specialized prompt types. A key reason is that Nemori avoids the trap of *message-wise* processing that many baselines fall into by using *episode* as its primary processing unit.
A finer-grained breakdown of Nemori’s costs is provided in Table [9](#A3.T9) of Appendix.

##### Response Generation.

**Table 4: Comparison of response generation cost on LoCoMo using gpt-4o-mini. Search denotes memory retrieval time; Total denotes end-to-end latency from receiving the question to completing the answer.**
| Method | LLM | Tokens | Search (ms) | Total (ms) |
| --- | --- | --- | --- | --- |
| FullContext | 72.3 | 23,653 | – | 5,806 |
| RAG-4096 | 30.2 | 3,430 | 544 | 2,884 |
| LangMem | 51.3 | 125 | 19,829 | 22,082 |
| Zep | 58.5 | 2,247 | 522 | 3,255 |
| Mem0 | 61.3 | 1,027 | 784 | 3,539 |
| A-MEM | 52.5 | 2,614 | 947 | 2,867 |
| MemoryOS | 54.5 | 1,560 | 9,910 | 15,220 |
| Nemori | 73.0 | 2,745 | 787 | 3,053 |

Table [4](#S4.T4) reports retrieval-time efficiency on LoCoMo with gpt-4o-mini. Here, Nemori uses 2,745 tokens on average, an 88% reduction compared with Full Context’s 23,653 tokens, while achieving slightly higher accuracy (73.0 vs. 72.3) and 47% lower total latency (3,053ms vs. 5,806ms).

### 4.4 Ablation Study (RQ3)

Table [5](#S4.T5) presents the ablation results. A finer-grained result is provided in Table [10](#A3.T10) of Appendix. We highlight the following observations:

##### Prediction-error-based vs. Direct Distillation.

The superior performance of w/o e over Nemori-s confirms the effectiveness of Nemori’s adaptive distillation design.
Both ablations discard the episodic database at response generation. The key difference is that w/o e generates responses using the semantic database from Nemori’s prediction-error-based distillation, while Nemori-s implements direct knowledge distillation over each incoming raw episode with the prompt in Appendix [D.2](#A4.SS2).
On gpt-4o-mini, prediction-error-based distillation achieves 65.0 vs. 52.0 for direct distillation (+25.0%); on gpt-4.1-mini, 74.9 vs. 65.5 (+14.4%).

**Table 5: Ablation study on LoCoMo. w/o Nemori = without Nemori; Nemori-s = semantic-only (direct distillation); w/o e = without episodic retrieval; w/o s = without semantic retrieval; w/o p = without adaptive partitioning (fixed 20-message chunks); Nemori = full framework. Mgmt: ✓ = with native management, ✗ = naive RAG detailed in Appendix [A.2](#A1.SS2).**
|  | Configuration | Mgmt | LLM | F1 | BLEU |
| --- | --- | --- | --- | --- | --- |
| gpt-4o-mini | w/o Nemori | – | 0.6 | 0.5 | 0.9 |
| Nemori-s | ✓ | 51.7 | 36.4 | 28.9 |  |
| ✗ | 52.0 | 36.6 | 29.1 |  |  |
| w/o e | ✓ | 64.6 | 46.2 | 37.1 |  |
| ✗ | 65.0 | 46.2 | 36.9 |  |  |
| w/o s | ✓ | 54.7 | 39.6 | 31.7 |  |
| w/o p | ✓ | 68.0 | 47.4 | 36.8 |  |
| Nemori | ✓ | 73.0 | 50.3 | 39.7 |  |
| gpt-4.1-mini | w/o Nemori | – | 1.2 | 1.6 | 1.5 |
| Nemori-s | ✓ | 66.0 | 41.4 | 34.9 |  |
| ✗ | 65.5 | 41.1 | 34.1 |  |  |
| w/o e | ✓ | 74.7 | 48.2 | 40.9 |  |
| ✗ | 74.9 | 48.1 | 40.7 |  |  |
| w/o s | ✓ | 76.9 | 50.0 | 42.9 |  |
| w/o p | ✓ | 75.7 | 48.1 | 40.9 |  |
| Nemori | ✓ | 80.8 | 52.1 | 45.0 |  |

Figure: Figure 2: Perfomance on LoCoMo with gpt-4.1-mini across observation window lengths $w=5-40$, where $w=20$ is the default setting in the main experiments.
Refer to caption: 2508.03341v4/x2.png

##### Native Management Contribution.

Toggling native management (✓ vs. ✗) shows minimal impact: 64.6 vs. 65.0 on gpt-4o-mini and 74.7 vs. 74.9 on gpt-4.1-mini.
This is expected, as LoCoMo rarely involves knowledge updates requiring consolidation.
We nevertheless retain native management for real-world deployment where such updates might be common.

##### Episodic–Semantic Complementarity.

Both memory types are indispensable.
Removing episodic retrieval (w/o e) drops performance from 73.0 to 65.0 ($-$11.0%) on gpt-4o-mini and from 80.8 to 74.9 ($-$7.3%) on gpt-4.1-mini.
Removing semantic retrieval (w/o s) drops performance from 73.0 to 54.7 ($-$25.1%) on gpt-4o-mini and from 80.8 to 76.9 ($-$4.8%) on gpt-4.1-mini.

##### Observation Window Length.

As shown in Figure [2](#S4.F2), Nemori’s performance remains stable across observation window lengths from 5 to 40, indicating that its design of message partitioning with integration is robust to this hyperparameter.
A finer-grained result is provided in Table [11](#A3.T11) of Appendix.

Figure: Figure 3: Effect of retrieval count $k$ on LLM Score for gpt-4o-mini (left) and gpt-4.1-mini (right). Dashed lines indicate the Full Context baseline. The annotated points mark the default setting ($k{=}10$) used in main experiments.
Refer to caption: 2508.03341v4/x3.png

**Table 6: Retrieval strategy ablation on LoCoMo using gpt-4.1-mini. N = narrative episodes; P = raw (partitioned) episodes. Index denotes the embedding source; Retrieve denotes the content returned to the LLM. When Retrieve = N, the top-2 narratives additionally include their raws, as described in Section [4.1](#S4.SS1). Bold row marks the default setting.**
| Index | Retrieve | LLM | F1 | BLEU |
| --- | --- | --- | --- | --- |
| N | N | 76.9 | 50.0 | 42.9 |
| P | N | 76.4 | 50.7 | 43.7 |
| N | P | 77.0 | 50.2 | 42.6 |
| P | P | 75.3 | 50.1 | 42.7 |

### 4.5 Retrieval Hyperparameter Analysis (RQ4)

We analyze two aspects of the retrieval configuration described in Section [3.4](#S3.SS4): the sensitivity to retrieval count $k$, and the choice of index–retrieve strategy.

##### Top-K Sensitivity.

Figure [3](#S4.F3) shows that performance rises sharply as $k$ increases from 2 to 10, then plateaus at a stable level that exceeds Full Context.
This suggests that a simple Top-K search strategy with mild retrieval count already saturates performance, indicating that Nemori effectively mitigates memory noise through its distillation process.
A finer-grained result is provided in Table [12](#A3.T12) of Appendix.

##### Index–Retrieve Strategy.

Table [6](#S4.T6) supports the representation prior and our implementation in Section [3.2.2](#S3.SS2.SSS2): holding the retrieved content fixed, narrative episode embeddings consistently outperform raw episode embeddings (76.9 vs. 76.4 when retrieving N; 77.0 vs. 75.3 when retrieving P).
A finer-grained breakdown is provided in Table [13](#A3.T13) in the Appendix.

**Table 7: Third-party management comparison. P=raw messages; $\mathcal{K}$=Nemori’s distilled semantic knowledge. Core=weighted average excluding Temporal.**
|  | System | Input | $\uparrow$LLM Score | $\downarrow$MemTokens |  |
| --- | --- | --- | --- | --- | --- |
|  | Average | Core |  |  |  |
| gpt-4o-mini | A-MEM | P | 52.5 | 52.6 | 397K |
| $\mathcal{K}$ | 50.9 | 55.8 | 142K |  |  |
| $\Delta$ | $\downarrow$3.0% | $\uparrow$6.1% | $\downarrow$64.3% |  |  |
| MemoryOS | P | 54.6 | 59.2 | 405K |  |
| $\mathcal{K}$ | 54.0 | 60.3 | 190K |  |  |
| $\Delta$ | $\downarrow$1.1% | $\uparrow$1.9% | $\downarrow$53.1% |  |  |
| gpt-4.1-mini | A-MEM | P | 61.4 | 60.4 | 498K |
| $\mathcal{K}$ | 59.0 | 64.1 | 243K |  |  |
| $\Delta$ | $\downarrow$3.9% | $\uparrow$6.1% | $\downarrow$51.3% |  |  |
| MemoryOS | P | 60.7 | 66.9 | 354K |  |
| $\mathcal{K}$ | 61.4 | 69.2 | 194K |  |  |
| $\Delta$ | $\uparrow$1.2% | $\uparrow$3.4% | $\downarrow$45.3% |  |  |

### 4.6 Third-Party Integration (RQ5)

Table [7](#S4.T7) evaluates Nemori as an adaptive distillation kernel for third-party memory systems.
Fed with semantic knowledge $\mathcal{K}$ instead of raw episodes (messages) $P$, both A-MEM and MemoryOS reduce storage by 45–64% while maintaining average performance ($\pm$4%), with core scores improving (+1.9% to +6.1%).
A finer-grained breakdown is provided in Table [14](#A3.T14) and Table [15](#A3.T15) in the Appendix.

### 4.7 Scalability Analysis (RQ6)

We evaluate on LongMemEval${}_{\text{S}}$ (105K), an order of magnitude longer than LoCoMo (9K).
As shown in Table [8](#S4.T8), Nemori outperforms Full Context by $+$16.7% on gpt-4o-mini and $+$13.7% on gpt-4.1-mini, a substantial increase from the marginal gains on LoCoMo ($+$1.0% and $+$0.2%).
The consistent improvements across both datasets suggest that Nemori captures general properties of interaction sequences rather than artifacts of a particular benchmark.
The widening gap further demonstrates that distillation becomes increasingly valuable as context grows: Full Context suffers from attention dilution over long inputs, while Nemori maintains focused retrieval with 95–96% fewer tokens.

**Table 8: Performance comparison on LongMemEvalS dataset. Nemori achieves higher accuracy while using 95–96% less context.**
|  | Question Type | Full-context | Nemori |
| --- | --- | --- | --- |
|  |  | (101K tok.) | (3.7–4.8K tok.) |
| gpt-4o-mini | Single-session Preference | 6.7 | 46.7 |
| Single-session Assistant | 89.3 | 83.9 |  |
| Temporal Reasoning | 42.1 | 61.7 |  |
| Multi-session | 38.3 | 51.1 |  |
| Knowledge Update | 78.2 | 61.5 |  |
| Single-session User | 78.6 | 88.6 |  |
| Average | 55.0 | 64.2 |  |
| gpt-4.1-mini | Single-session Preference | 16.7 | 86.7 |
| Single-session Assistant | 98.2 | 92.9 |  |
| Temporal Reasoning | 60.2 | 72.2 |  |
| Multi-session | 51.1 | 55.6 |  |
| Knowledge Update | 76.9 | 79.5 |  |
| Single-session User | 85.7 | 90.0 |  |
| Average | 65.6 | 74.6 |  |

## 5 Conclusion

Inspired by cognitive ideas, we propose Nemori, a training-free framework that adaptively assesses the future utility of an agent’s experience at the distillation stage, where prediction error deserves retention as memory.
Guided by three priors about structure, representation and distillation of interaction sequences, Nemori’s cascading modules work in coordination. Its episode-centric design enhances token efficiency, and its management-agnostic design allows it to serve as a distillation layer for downstream memory systems.
Our results suggest that being agentic need not imply being heuristic: observations over data properties can establish a data-driven space, increasingly important as agents requiring long-term behavioral consistency usually operate beyond human curation.

## Limitations

We acknowledge two limitations.
*First*, Nemori focuses on distillation and adopts naive strategies for management and retrieval. This simplicity suffices for the benchmarks studied here, but may become a bottleneck for tasks that demand more sophisticated reasoning over memory. The performance we report should be understood as reflecting this design scope rather than the ceiling of what a complete memory system could achieve.
*Second*, the interfaces we define are currently conceptual, and due to the lack of standardized protocols in the area, concrete integration still requires case-by-case implementation.

## Acknowledgments

The authors would first like to thank the anonymous reviewers for their constructive feedback.
We are also grateful to Shanda Group for providing the resources that supported this work.
We thank Prof. Weiguo Zheng for his advice on highlighting our core contributions, Prof. Jiaye Teng for his advice on the paper’s overall structure, and Prof. Yixuan Qiu for his detailed suggestions on our presentation.
We also thank Huaqing Zhang and Chenrui Wang for their generous help in reviewing an earlier version of this paper.

Wenquan Ma thanks the transferable knowledge and skills he gained in Prof. Zheng’s group and in his previous research under Prof. Teng’s supervision.
Jiayan Nan thanks Yize Chen for the support and help in his work and research.
Finally, the authors thank each other, and look forward to more opportunities for collaboration in the future.

## References

- P. Anokhin, N. Semenov, A. Y. Sorokin, D. Evseev, A. Kravchenko, M. Burtsev, and E. Burnaev (2025)
AriGraph: learning knowledge graph world models with episodic memory for LLM agents.
In Proceedings of the Thirty-Fourth International Joint Conference on
Artificial Intelligence, IJCAI 2025, Montreal, Canada, August 16-22,
2025,
pp. 12–20.
External Links: [Link](https://doi.org/10.24963/ijcai.2025/2),
[Document](https://dx.doi.org/10.24963/IJCAI.2025/2)
Cited by: [Table 1](#S1.T1.1.7.7.1).
- H. Chase (2022)
LangChain.
Note: [https://github.com/langchain-ai/langchain](https://github.com/langchain-ai/langchain)Accessed: 2025-12-31
Cited by: [§4.1](#S4.SS1.SSS0.Px2.p1.1).
- P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav (2025)
Mem0: building production-ready AI agents with scalable long-term memory.
CoRR abs/2504.19413.
External Links: [Link](https://doi.org/10.48550/arXiv.2504.19413),
[Document](https://dx.doi.org/10.48550/ARXIV.2504.19413),
2504.19413
Cited by: [Table 1](#S1.T1.1.12.12.1),
[§1](#S1.p2.1),
[§2.1](#S2.SS1.p1.1),
[§4.1](#S4.SS1.SSS0.Px2.p1.1).
- A. Clark (2013)
Whatever next? predictive brains, situated agents, and the future of cognitive science.
Behavioral and brain sciences 36 (3), pp. 181–204.
Cited by: [§1](#S1.p3.1),
[§2.2](#S2.SS2.p1.1),
[§3](#S3.p1.1).
- J. Fang, X. Deng, H. Xu, Z. Jiang, Y. Tang, Z. Xu, S. Deng, Y. Yao, M. Wang, S. Qiao, H. Chen, and N. Zhang (2025)
LightMem: lightweight and efficient memory-augmented generation.
CoRR abs/2510.18866.
External Links: [Link](https://doi.org/10.48550/arXiv.2510.18866),
[Document](https://dx.doi.org/10.48550/ARXIV.2510.18866),
2510.18866
Cited by: [§4.3](#S4.SS3.SSS0.Px1.p1.1).
- K. Friston (2010)
The free-energy principle: a unified brain theory?.
Nature Reviews Neuroscience 11 (2), pp. 127–138.
External Links: ISSN 1471-0048,
[Link](https://doi.org/10.1038/nrn2787),
[Document](https://dx.doi.org/10.1038/nrn2787)
Cited by: [§1](#S1.p3.1),
[§2.2](#S2.SS2.p1.1),
[§3](#S3.p1.1).
- L. Huang, H. Lan, Z. Sun, C. Shi, and T. Bai (2024)
Emotional RAG: enhancing role-playing agents through emotional retrieval.
In IEEE International Conference on Knowledge Graph, ICKG 2023, Shanghai,
China, December 1-2, 2023, V. S. Sheng, C. Hicks, C. Ling, V. Raghavan, and X. Wu (Eds.),
pp. 120–127.
External Links: [Link](https://doi.org/10.1109/ICKG63256.2024.00023),
[Document](https://dx.doi.org/10.1109/ICKG63256.2024.00023)
Cited by: [Table 1](#S1.T1.1.11.11.1),
[§1](#S1.p2.1),
[§2.1](#S2.SS1.p1.1).
- J. Kang, M. Ji, Z. Zhao, and T. Bai (2025)
Memory OS of AI agent.
In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.),
Suzhou, China, pp. 25972–25981.
External Links: [Link](https://aclanthology.org/2025.emnlp-main.1318/),
[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1318),
ISBN 979-8-89176-332-6
Cited by: [§A.3](#A1.SS3.p1.1),
[Table 1](#S1.T1.1.5.5.1),
[§1](#S1.p2.1),
[§2.1](#S2.SS1.p1.1),
[§4.1](#S4.SS1.SSS0.Px2.p1.1).
- P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela (2020)
Retrieval-augmented generation for knowledge-intensive NLP tasks.
In Advances in Neural Information Processing Systems 33: Annual Conference
on Neural Information Processing Systems 2020, NeurIPS 2020, December
6-12, 2020, virtual, H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, and H. Lin (Eds.),
External Links: [Link](https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html)
Cited by: [Table 1](#S1.T1.1.2.2.2),
[§2.1](#S2.SS1.p1.1),
[§4.1](#S4.SS1.SSS0.Px2.p1.1).
- H. Li, C. Yang, A. Zhang, Y. Deng, X. Wang, and T. Chua (2025a)
Hello again! LLM-powered personalized agent for long-term dialogue.
In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), L. Chiruzzo, A. Ritter, and L. Wang (Eds.),
Albuquerque, New Mexico, pp. 5259–5276.
External Links: [Link](https://aclanthology.org/2025.naacl-long.272/),
[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.272),
ISBN 979-8-89176-189-6
Cited by: [Table 1](#S1.T1.1.13.13.1).
- R. Li, Z. Zhang, X. Bo, Z. Tian, X. Chen, Q. Dai, Z. Dong, and R. Tang (2025b)
CAM: a constructivist view of agentic memory for LLM-based reading comprehension.
In The Thirty-ninth Annual Conference on Neural Information Processing Systems,
External Links: [Link](https://openreview.net/forum?id=ACSOnSHiWe)
Cited by: [Table 1](#S1.T1.1.6.6.1).
- N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang (2024)
Lost in the middle: how language models use long contexts.
Transactions of the Association for Computational Linguistics 12, pp. 157–173.
External Links: [Link](https://aclanthology.org/2024.tacl-1.9/),
[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00638)
Cited by: [§1](#S1.p1.1).
- A. Maharana, D. Lee, S. Tulyakov, M. Bansal, F. Barbieri, and Y. Fang (2024)
Evaluating very long-term conversational memory of LLM agents.
In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.),
Bangkok, Thailand, pp. 13851–13870.
External Links: [Link](https://aclanthology.org/2024.acl-long.747/),
[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.747)
Cited by: [§4.1](#S4.SS1.SSS0.Px1.p1.1).
- J. L. McClelland, B. L. McNaughton, and R. C. O’Reilly (1995)
Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory..
Psychological review 102 (3), pp. 419.
Cited by: [§1](#S1.p3.1),
[§3](#S3.p1.1).
- C. Packer, V. Fang, S. G. Patil, K. Lin, S. Wooders, and J. E. Gonzalez (2023)
MemGPT: towards llms as operating systems.
CoRR abs/2310.08560.
External Links: [Link](https://doi.org/10.48550/arXiv.2310.08560),
[Document](https://dx.doi.org/10.48550/ARXIV.2310.08560),
2310.08560
Cited by: [Table 1](#S1.T1.1.3.3.2),
[§2.1](#S2.SS1.p1.1).
- Z. Pan, Q. Wu, H. Jiang, X. Luo, H. Cheng, D. Li, Y. Yang, C. Lin, H. V. Zhao, L. Qiu, and J. Gao (2025)
SeCom: on memory construction and retrieval for personalized conversational agents.
In The Thirteenth International Conference on Learning Representations,
ICLR 2025, Singapore, April 24-28, 2025,
External Links: [Link](https://openreview.net/forum?id=xKDZAW0He3)
Cited by: [Table 1](#S1.T1.1.14.14.1).
- J. S. Park, J. C. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein (2023)
Generative agents: interactive simulacra of human behavior.
In Proceedings of the 36th Annual ACM Symposium on User Interface Software
and Technology, UIST 2023, San Francisco, CA, USA, 29 October 2023-
1 November 2023, S. Follmer, J. Han, J. Steimle, and N. H. Riche (Eds.),
pp. 2:1–2:22.
External Links: [Link](https://doi.org/10.1145/3586183.3606763),
[Document](https://dx.doi.org/10.1145/3586183.3606763)
Cited by: [Table 1](#S1.T1.1.10.10.2),
[§1](#S1.p2.1),
[§2.1](#S2.SS1.p1.1).
- R. P. Rao and D. H. Ballard (1999)
Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects.
Nature neuroscience 2 (1), pp. 79–87.
Cited by: [§1](#S1.p3.1),
[§2.2](#S2.SS2.p1.1),
[§3](#S3.p1.1).
- P. Rasmussen, P. Paliychuk, T. Beauvais, J. Ryan, and D. Chalef (2025)
Zep: A temporal knowledge graph architecture for agent memory.
CoRR abs/2501.13956.
External Links: [Link](https://doi.org/10.48550/arXiv.2501.13956),
[Document](https://dx.doi.org/10.48550/ARXIV.2501.13956),
2501.13956
Cited by: [Table 1](#S1.T1.1.9.9.1),
[§4.1](#S4.SS1.SSS0.Px2.p1.1),
[§4.1](#S4.SS1.SSS0.Px3.p1.1).
- D. Wu, H. Wang, W. Yu, Y. Zhang, K. Chang, and D. Yu (2025)
LongMemEval: benchmarking chat assistants on long-term interactive memory.
In The Thirteenth International Conference on Learning Representations,
ICLR 2025, Singapore, April 24-28, 2025,
External Links: [Link](https://openreview.net/forum?id=pZiyCaVuti)
Cited by: [§4.1](#S4.SS1.SSS0.Px1.p1.1).
- W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang (2025)
A-mem: agentic memory for LLM agents.
In The Thirty-ninth Annual Conference on Neural Information Processing Systems,
External Links: [Link](https://openreview.net/forum?id=FiM0M8gcct)
Cited by: [§A.3](#A1.SS3.p1.1),
[Table 1](#S1.T1.1.8.8.1),
[§1](#S1.p2.1),
[§2.1](#S2.SS1.p1.1),
[§4.1](#S4.SS1.SSS0.Px2.p1.1).
- W. Zhong, L. Guo, Q. Gao, H. Ye, and Y. Wang (2024)
MemoryBank: enhancing large language models with long-term memory.
In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI
2024, Thirty-Sixth Conference on Innovative Applications of Artificial
Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances
in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver,
Canada, M. J. Wooldridge, J. G. Dy, and S. Natarajan (Eds.),
pp. 19724–19731.
External Links: [Link](https://doi.org/10.1609/aaai.v38i17.29946),
[Document](https://dx.doi.org/10.1609/AAAI.V38I17.29946)
Cited by: [Table 1](#S1.T1.1.4.4.1),
[§1](#S1.p2.1),
[§2.1](#S2.SS1.p1.1).

## Appendix A Implementations of Management

This appendix details three instantiations of the management interfaces defined in Section [3.3](#S3.SS3), demonstrating Nemori’s architectural flexibility.

### A.1 Conceptual Model: Flat Summarization

This configuration only serves as a conceptual illustration to aid understanding. Here, $\mathcal{M}$ maintains a single monolithic summary rather than a structured database.

##### Context Evocation.

The global summary $\mathcal{S}_{sum}$ is returned as constant context:

$$ $\operatorname{Evoke}(M_{in},\mathcal{M}):\quad\mathcal{S}_{in}\leftarrow\mathcal{S}_{sum}$ $$

##### Knowledge Consolidation.

New insights are directly merged into the summary:

$$ $\mathcal{S}_{sum}\leftarrow f_{\text{LLM}}(\mathcal{P}_{\text{sum}}\parallel\mathcal{K}_{in}\parallel\mathcal{S}_{sum})$ $$

where $\mathcal{P}_{\text{sum}}$ instructs the LLM to merge distilled insights into the summary.

### A.2 Variant: Naive RAG

This configuration serves as an ablation in Section [4.4](#S4.SS4), applying no management to semantic memory. Distilled insights are directly stored and retrieved via similarity search, without conflict resolution.

##### Context Evocation.

For each input $M_{in}$, the interface retrieves top-$K_{s}$ semantically similar entries:

$$ $\mathcal{S}_{in}\leftarrow\operatorname{Top-}K_{s}\left(S_{r}\in\mathcal{D}_{s}\mid\operatorname{sim}(\mathbf{v}_{in},\mathbf{u}_{r})>\tau\right)$ $$

##### Knowledge Consolidation.

Each distilled insight $k_{q}\in\mathcal{K}_{in}$ is simply embedded and appended:

$$ $\mathcal{D}_{s}\leftarrow\mathcal{D}_{s}\cup\{(k_{q},f_{\text{emb}}(k_{q}))\}$ $$

No management like conflict detection or merging is performed. This contrasts with our native implementation (Section [3.3](#S3.SS3)), which applies the full consolidation logic with new/merge/conflict resolution.

### A.3 External Integration: Third-Party Systems

Nemori can be implemented with third-party management systems (Xu et al., 2025; Kang et al., 2025) by intercepting their context assembly and injecting distilled content.

##### Context Evocation.

Most of the memory systems fundamentally operate by conditioning response generation on related context.
We intercept this context buffer $\tilde{\mathcal{B}}$, assembled by the host’s management logic for query $M_{in}$, and repurpose it as the basis for prediction: instead of generating a response, we use it to synthesize an anticipatory schema of what should have occurred:

$$ $\mathcal{S}_{in}\leftarrow\tilde{\mathcal{B}}$ $$

##### Knowledge Consolidation.

Each distilled insight $k_{q}\in\mathcal{K}_{in}$ is injected as an independent message into the host’s input sequence, allowing the external system to manage it natively.
Notably, most of the memory systems discussed in this paper are designed to process explicit factual knowledge, making Nemori’s distilled semantic memory a suitable input.

Figure: Algorithm 1 Nemori Memory Distillation

Figure: Algorithm 2 Nemori Response Generation

## Appendix B Case Study

This section presents two representative cases from the main text, highlighting how Nemori supports temporal reasoning and open-domain question answering.

### B.1 Temporal Reasoning

To illustrate how Nemori enhances response quality, we provide a representative case from the LoCoMo dataset.

Question: “When did Jon receive mentorship?”

Challenge: The original conversation contains relative temporal references like “yesterday” without explicit dates, requiring temporal reasoning.

Full Context baseline: Confused by the term “yesterday” in the raw dialogue, the model incorrectly answered with the conversation date (June 16).

Nemori: Retrieved both the relevant episodic memory (preserving conversational context) and a semantic memory that had already distilled the temporal information into explicit fact: “Jon was mentored on June 15, 2023.” By combining episodic context with pre-reasoned semantic knowledge, Nemori transforms complex reasoning into simple fact retrieval.

Insight: This demonstrates the capability of “reasoning during memory formation.” The prediction error highlight that the specific date is unexpected given prior knowledge, prompting its distillation as semantic memory.

### B.2 Open Domain

On the Open Domain subset, Nemori’s LLM score is slightly below the strongest memory system baseline, with gaps of 6.8% under gpt-4.1-mini (56.3 vs. 60.4) and 3.8% under gpt-4o-mini (45.8 vs. 47.6). We note that this subset is not a pure measure of the memory procedure’s effectiveness, specifically:

In LoCoMo, many such questions are not directly answerable from the original conversation history alone; instead, they require the backbone model to recognize a conversational description and map it to an item of general world knowledge. As a result, performance in this category depends not only on memory quality, but also on the model’s prior knowledge.

A representative example is the question: “What is the game with different colored cards that John was talking about with James?” The gold answer is “UNO”, but the dialogue itself never explicitly names UNO. Instead, the transcript only states that the players discussed a game with multi-colored cards and matching by color or number, while also noting that the speaker had forgotten its name. Accordingly, Nemori’s episodic memory preserves this conversational evidence, and the semantic memory distills the same game description, but neither memory can inject the missing lexical label if it is absent from the interaction history. In such cases, whether the final answer becomes “UNO” depends largely on the backbone model’s ability to recognize the description from prior knowledge, rather than on a failure of memory distillation or retrieval.

## Appendix C Additional Experiment Results

This appendix provides detailed experimental results that supplement the main paper. All experiments use the setup described in Section [4.1](#S4.SS1).

**Table 9: Component-wise breakdown of Nemori’s memory construction cost on LoCoMo with gpt-4o-mini.**
| Component | Input (k) | Output (k) | Total (k) | Ratio |
| --- | --- | --- | --- | --- |
| Partition (§[3.2.1](#S3.SS2.SSS1)) | 44.7 | 4.6 | 49.3 | 15.3% |
| Narration (§[3.2.2](#S3.SS2.SSS2)) | 99.8 | 23.9 | 123.6 | 38.3% |
| Integration (§[3.2.3](#S3.SS2.SSS3)) | 43.8 | 8.4 | 52.2 | 16.2% |
| Distillation (§[3.3](#S3.SS3)) | 88.9 | 8.8 | 97.7 | 30.3% |

**Table 10: Category-wise Ablation study on LoCoMo. Nemori-s = semantic-only (direct distillation); w/o e = without episodic retrieval; w/o s = without semantic retrieval; w/o p = without adaptive partitioning (fixed 20-message chunks); Nemori = full framework. Mgmt: ✓ = with native management, ✗ = naive RAG detailed in Section [A.2](#A1.SS2).**
|  | Configuration | Mgmt | Temporal Reasoning | Open Domain | Multi-Hop | Single-Hop | Overall |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU |  |  |
| gpt-4o-mini | Nemori-s | ✓ | 33.3 | 36.8 | 31.1 | 49.0 | 24.4 | 18.6 | 47.9 | 30.5 | 20.2 | 60.3 | 39.7 | 32.1 | 51.7 | 36.4 | 28.9 |
| ✗ | 32.7 | 35.9 | 30.4 | 40.6 | 21.8 | 17.0 | 47.5 | 31.3 | 20.7 | 62.1 | 40.3 | 32.7 | 52.0 | 36.6 | 29.1 |  |  |
| w/o e | ✓ | 57.9 | 53.0 | 44.6 | 53.1 | 26.5 | 19.6 | 57.8 | 35.5 | 24.4 | 70.8 | 49.4 | 40.4 | 64.6 | 46.2 | 37.1 |  |
| ✗ | 56.7 | 52.8 | 44.8 | 54.2 | 27.8 | 20.6 | 59.9 | 36.5 | 24.8 | 71.1 | 48.9 | 39.8 | 65.0 | 46.2 | 36.9 |  |  |
| w/o s | ✓ | 32.7 | 38.9 | 33.0 | 42.7 | 22.2 | 17.0 | 53.9 | 33.0 | 22.2 | 64.7 | 44.1 | 36.1 | 54.7 | 39.6 | 31.7 |  |
| w/o p | ✓ | 56.7 | 52.7 | 43.4 | 45.8 | 25.0 | 19.3 | 59.9 | 36.3 | 23.5 | 77.5 | 51.7 | 40.7 | 68.0 | 47.4 | 36.8 |  |
| Nemori | ✓ | 67.6 | 57.3 | 47.6 | 45.8 | 23.9 | 18.5 | 61.7 | 38.1 | 26.0 | 81.9 | 54.8 | 43.8 | 73.0 | 50.3 | 39.7 |  |
| gpt-4.1-mini | Nemori-s | ✓ | 46.4 | 42.2 | 33.7 | 49.0 | 26.5 | 20.7 | 67.4 | 36.2 | 28.8 | 74.9 | 44.5 | 39.0 | 66.0 | 41.4 | 34.9 |
| ✗ | 47.0 | 42.5 | 32.5 | 50.0 | 28.5 | 22.7 | 70.6 | 38.9 | 29.9 | 72.7 | 42.8 | 37.4 | 65.5 | 41.1 | 34.1 |  |  |
| w/o e | ✓ | 63.2 | 49.6 | 41.1 | 52.1 | 27.2 | 21.2 | 72.7 | 38.9 | 29.1 | 82.4 | 53.2 | 47.0 | 74.7 | 48.2 | 40.9 |  |
| ✗ | 65.4 | 51.0 | 42.5 | 56.3 | 29.1 | 23.0 | 70.2 | 39.0 | 29.5 | 82.2 | 52.2 | 45.8 | 74.9 | 48.1 | 40.7 |  |  |
| w/o s | ✓ | 73.5 | 54.3 | 46.9 | 55.2 | 26.9 | 21.3 | 73.1 | 41.9 | 32.5 | 81.9 | 53.7 | 47.3 | 76.9 | 50.0 | 42.9 |  |
| w/o p | ✓ | 67.3 | 53.4 | 45.0 | 52.1 | 24.7 | 19.5 | 71.3 | 40.1 | 30.7 | 83.1 | 51.4 | 45.2 | 75.7 | 48.1 | 40.9 |  |
| Nemori | ✓ | 77.3 | 58.7 | 50.7 | 56.3 | 31.7 | 25.1 | 74.8 | 40.8 | 31.7 | 87.0 | 55.7 | 49.5 | 80.8 | 52.1 | 45.0 |  |

**Table 11: Performance across different observation window lengths on LoCoMo dataset with gpt-4.1-mini. w=20 (bold) is the default setting used in main experiments.**
| $w$ | Temporal | Open Domain | Multi-Hop | Single-Hop | Overall |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU |  |
| 5 | 77.0 | 57.6 | 49.6 | 59.4 | 30.0 | 24.9 | 73.8 | 43.0 | 34.0 | 86.3 | 55.0 | 48.5 | 80.4 | 51.8 | 44.6 |
| 10 | 77.6 | 59.0 | 50.4 | 57.3 | 29.5 | 24.7 | 77.0 | 44.0 | 34.3 | 85.7 | 54.5 | 48.1 | 80.7 | 52.0 | 44.6 |
| 20 | 77.3 | 58.7 | 50.7 | 56.3 | 31.7 | 25.1 | 74.8 | 40.8 | 31.7 | 87.0 | 55.7 | 49.5 | 80.8 | 52.1 | 45.0 |
| 30 | 76.6 | 57.7 | 49.6 | 60.4 | 32.2 | 26.1 | 79.4 | 45.0 | 34.8 | 86.0 | 55.4 | 48.8 | 81.2 | 52.5 | 45.0 |
| 40 | 76.3 | 58.6 | 50.4 | 54.2 | 27.1 | 21.4 | 77.3 | 42.8 | 34.0 | 86.4 | 55.0 | 48.4 | 80.7 | 51.8 | 44.5 |

**Table 12: Category-wise breakdown of retrieval count $k$ on LoCoMo for gpt-4o-mini and gpt-4.1-mini. Semantic memory count is fixed at $m=2k$. Bold rows mark the default setting ($k{=}10$) used in main experiments.**
|  | $k$ | Temporal | Open Domain | Multi-Hop | Single-Hop | Overall |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU |  |
| gpt-4o-mini | 2 | 62.3 | 55.3 | 46.5 | 41.7 | 20.9 | 15.4 | 55.0 | 33.7 | 21.8 | 75.0 | 51.1 | 40.9 | 66.6 | 46.9 | 37.0 |
| 5 | 64.5 | 56.7 | 47.4 | 47.9 | 24.9 | 19.2 | 62.4 | 36.7 | 25.0 | 79.1 | 53.3 | 42.4 | 71.0 | 49.2 | 38.8 |  |
| 10 | 67.6 | 57.3 | 47.6 | 45.8 | 23.9 | 18.5 | 61.7 | 38.1 | 26.0 | 81.9 | 54.8 | 43.8 | 73.0 | 50.3 | 39.7 |  |
| 15 | 68.9 | 58.6 | 48.4 | 44.8 | 24.5 | 19.0 | 61.4 | 37.0 | 25.4 | 83.1 | 54.9 | 43.6 | 73.8 | 50.5 | 39.8 |  |
| 20 | 67.9 | 57.4 | 47.7 | 45.8 | 24.3 | 18.9 | 63.5 | 38.2 | 25.9 | 82.8 | 54.7 | 43.1 | 73.8 | 50.4 | 39.4 |  |
| 30 | 68.2 | 58.9 | 48.4 | 45.8 | 23.9 | 18.8 | 64.5 | 37.7 | 25.6 | 83.2 | 55.0 | 43.3 | 74.4 | 50.7 | 39.6 |  |
| gpt-4.1-mini | 2 | 68.5 | 52.8 | 45.5 | 52.1 | 25.7 | 20.5 | 64.5 | 38.2 | 28.3 | 80.6 | 51.7 | 45.6 | 73.4 | 47.8 | 40.9 |
| 5 | 74.1 | 56.7 | 48.9 | 55.2 | 28.8 | 22.6 | 73.1 | 41.9 | 32.3 | 86.1 | 54.4 | 48.1 | 79.3 | 51.0 | 43.8 |  |
| 10 | 77.3 | 58.7 | 50.7 | 56.3 | 31.7 | 25.1 | 74.8 | 40.8 | 31.7 | 87.0 | 55.7 | 49.5 | 80.8 | 52.1 | 45.0 |  |
| 15 | 80.1 | 59.6 | 51.7 | 57.3 | 31.1 | 24.3 | 77.7 | 42.6 | 33.2 | 88.2 | 55.9 | 49.5 | 82.7 | 52.7 | 45.4 |  |
| 20 | 79.8 | 59.4 | 51.1 | 58.3 | 30.3 | 24.4 | 81.2 | 45.3 | 35.5 | 88.5 | 55.6 | 49.1 | 83.4 | 52.9 | 45.5 |  |
| 30 | 80.4 | 60.0 | 52.0 | 60.4 | 29.9 | 23.0 | 79.4 | 44.3 | 35.0 | 88.4 | 56.1 | 49.5 | 83.3 | 53.1 | 45.7 |  |

**Table 13: Category-wise breakdown of retrieval strategy ablation on LoCoMo using gpt-4.1-mini. N = narrative episodes; P = raw (partitioned) episodes. Index denotes the embedding source; Retrieve denotes the content returned to the LLM. When Retrieve = N, the top-2 narratives additionally include their raws, as described in Section [4.1](#S4.SS1). Bold row marks the default setting.**
| Index | Retrieve | Temporal | Open Domain | Multi-Hop | Single-Hop | Overall |  |  |  |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU | LLM | F1 | BLEU |  |  |
| N | N | 72.9 | 55.7 | 48.3 | 55.2 | 26.0 | 20.4 | 72.3 | 41.8 | 32.8 | 82.3 | 53.3 | 47.1 | 76.9 | 50.0 | 42.9 |
| P | N | 71.7 | 56.5 | 49.0 | 47.9 | 24.0 | 19.6 | 72.3 | 40.5 | 30.8 | 82.8 | 55.0 | 48.7 | 76.4 | 50.7 | 43.7 |
| N | P | 73.8 | 43.4 | 35.8 | 55.2 | 27.3 | 21.5 | 72.0 | 43.2 | 32.4 | 82.4 | 57.8 | 51.0 | 77.0 | 50.2 | 42.6 |
| P | P | 69.5 | 42.5 | 35.2 | 51.0 | 22.6 | 18.2 | 67.0 | 40.2 | 30.5 | 83.0 | 59.4 | 52.4 | 75.3 | 50.1 | 42.7 |

**Table 14: LLM Score comparison of third-party management systems with different input sources on LoCoMo. N=raw conversation; $\mathcal{K}$=Nemori’s distilled semantic memory. Core=weighted average excluding Temporal.**
| Model | System | Input | Temp | Open | Multi | Single | Average | Core |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| gpt-4o-mini | A-MEM | N | 54.2 | 22.9 | 43.6 | 58.2 | 52.5 | 52.6 |
| $\mathcal{K}$ | 33.6 | 38.5 | 50.4 | 59.1 | 50.9 | 55.8 |  |  |
| $\Delta$ | $\downarrow$38.0% | $\uparrow$68.1% | $\uparrow$15.4% | $\uparrow$1.6% | $\downarrow$3.0% | $\uparrow$6.1% |  |  |
| MemoryOS | N | 38.0 | 45.8 | 52.5 | 62.5 | 54.6 | 59.2 |  |
| $\mathcal{K}$ | 30.8 | 44.8 | 58.5 | 62.4 | 54.0 | 60.3 |  |  |
| $\Delta$ | $\downarrow$18.9% | $\downarrow$2.2% | $\uparrow$11.5% | $\downarrow$0.2% | $\downarrow$1.1% | $\uparrow$1.9% |  |  |
| gpt-4.1-mini | A-MEM | N | 66.7 | 37.5 | 55.7 | 64.0 | 61.4 | 60.4 |
| $\mathcal{K}$ | 41.1 | 41.7 | 58.2 | 68.0 | 59.0 | 64.1 |  |  |
| $\Delta$ | $\downarrow$38.4% | $\uparrow$11.1% | $\uparrow$4.5% | $\uparrow$6.3% | $\downarrow$3.9% | $\uparrow$6.1% |  |  |
| MemoryOS | N | 37.7 | 60.4 | 62.4 | 68.9 | 60.7 | 66.9 |  |
| $\mathcal{K}$ | 32.7 | 58.3 | 62.4 | 72.3 | 61.4 | 69.2 |  |  |
| $\Delta$ | $\downarrow$13.3% | $\downarrow$3.5% | – | $\uparrow$5.0% | $\uparrow$1.2% | $\uparrow$3.4% |  |  |

**Table 15: Memory storage comparison of third-party management systems with different input sources. N = raw conversation; $\mathcal{K}$ = Nemori’s distilled semantic memory. Entries denotes the number of memory entries defined by each system’s own storage format (comparable within but not across systems). Tokens and Chars are measured by concatenating all entries. $\downarrow$ indicates reduction.**
| Model | System | Input | Tokens | Chars | Entries |
| --- | --- | --- | --- | --- | --- |
| gpt-4o-mini | A-MEM | N | 396,812 | 2,475,511 | 5,882 |
| $\mathcal{K}$ | 141,682 | 820,025 | 2,725 |  |  |
| $\Delta$ | $\downarrow$64.3% | $\downarrow$66.9% | $\downarrow$53.7% |  |  |
| MemoryOS | N | 404,611 | 1,956,432 | 3,014 |  |
| $\mathcal{K}$ | 189,662 | 927,678 | 2,613 |  |  |
| $\Delta$ | $\downarrow$53.1% | $\downarrow$52.6% | $\downarrow$13.3% |  |  |
| gpt-4.1-mini | A-MEM | N | 498,234 | 3,811,770 | 5,882 |
| $\mathcal{K}$ | 242,801 | 1,459,777 | 2,676 |  |  |
| $\Delta$ | $\downarrow$51.3% | $\downarrow$61.7% | $\downarrow$54.5% |  |  |
| MemoryOS | N | 354,463 | 1,712,305 | 3,017 |  |
| $\mathcal{K}$ | 193,744 | 1,023,709 | 2,383 |  |  |
| $\Delta$ | $\downarrow$45.3% | $\downarrow$40.2% | $\downarrow$21.0% |  |  |

##### Memory Construction Cost (Table 9 )

Finer-grained results of Section [4.3](#S4.SS3). The main cost comes from the narrative episode generation (38.3%) and semantic knowledge distillation (30.3%).

##### Ablation Study (Table 10 ).

Finer-grained results of Section [4.4](#S4.SS4).
Prediction-error-based distillation consistently outperforms direct knowledge distillation across categories.
The improvement is most pronounced in Temporal Reasoning, from 33.3 to 57.9 (+73.9%) on gpt-4o-mini and from 46.4 to 63.2 (+36.2%) on gpt-4.1-mini, where prediction-error-based distillation effectively identifies and transforms time-sensitive information.

##### Observation Window Length (Table 11 ).

Finer-grained results of Figure [2](#S4.F2).
Overall scores remain stable ($\pm$1%) across window lengths from 5 to 40.
Category-level variation is likewise small, confirming that Nemori’s design of message partitioning with integration is robust to this hyperparameter.

##### Top-K Sensitivity (Table 12 ).

Performance rises sharply as $k$ increases from 2 to 10, then plateaus. Strongest average performance is achieved at $k$=15$\sim$20, but $k$=10 provides 97% of peak performance with lower computational cost.

##### Retrieval Strategy (Table 13 ).

The N$\rightarrow$P configuration achieves a marginally higher LLM score (77.0 vs. 76.9), as raw text preserves factual details for answer generation.
We default to N$\rightarrow$N for simplicity, since the difference is negligible and narrative retrieval avoids returning lengthy raw episodes.

##### Third-Party Management (Tables 14 and 15 ).

Using Nemori’s semantic memory as input reduces storage by 45–64% while improving Core scores by 1.9–6.1%, demonstrating that distilled memory provides a compact yet information-rich representation suitable for downstream management systems.

## Appendix D Prompt Templates

This appendix provides the complete prompt templates used in Nemori’s pipeline.

### D.1 Core Distillation Prompts

This subsection presents the prompts for the main distillation modules described in Section [3.2](#S3.SS2) and Section [3.3](#S3.SS3), instantiated with our native management implementation.

#### D.1.1 Local Message Partitioning Prompt ( 𝒫 par \mathcal{P}_{\text{par}} )

#### D.1.2 Narrative Episode Generation Prompt ( 𝒫 nar \mathcal{P}_{\text{nar}} )

#### D.1.3 Optimal Candidate Identification Prompt ( 𝒫 sel \mathcal{P}_{\text{sel}} )

#### D.1.4 Episodic Integration Prompt ( 𝒫 int \mathcal{P}_{\text{int}} )

#### D.1.5 Anticipatory Schema Synthesis Prompt ( 𝒫 ant \mathcal{P}_{\text{ant}} )

#### D.1.6 Prediction Error Distillation Prompt ( 𝒫 dis \mathcal{P}_{\text{dis}} )

#### D.1.7 Semantic Consolidation Prompt ( 𝒫 con \mathcal{P}_{\text{con}} )

### D.2 Direct Distillation Prompt ( Nemori -s)

This prompt corresponds to the Nemori-s configuration in our ablation study in Section [4.4](#S4.SS4), which performs direct knowledge distillation without prediction-error-based distillation.

### D.3 Response Generation Prompt ( 𝒫 ans \mathcal{P}_{\text{ans}} )

This unified prompt is used for response generation across all evaluation tasks on both LoCoMo and LongMemEval${}_{\text{S}}$ datasets.

### D.4 LLM-as-Judge Prompts

#### D.4.1 LoCoMo

LoCoMo uses a single unified evaluation prompt for all question categories.

#### D.4.2 LongMemEval S {}_{\text{S}}

In contrast to LoCoMo’s unified prompt, LongMemEval${}_{\text{S}}$ uses task-specific prompts for evaluation. We present the four variants below.