##### Report GitHub Issue × Title: Content selection saved. Describe the issue below: Description: Submit without GitHub Submit in GitHub [arXiv logoBack to arXiv](/) [License: arXiv.org perpetual non-exclusive license](https://info.arxiv.org/help/license/index.html#licenses-available) arXiv:2605.00814v1 [cs.CV] 01 May 2026 Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs ============================================================================= Siyuan Huang1,2, Xiaoye Qu1 , Yafu Li3, Tong Zhu1, Zefeng He4 Muxin Fu5, Daizong Liu6, Wei-Long Zheng2, Yu Cheng3† 1Shanghai AI Laboratory,2Shanghai Jiao Tong University, 3The Chinese University of Hong Kong,4Nanjing University, 5Tongji University,6Wuhan UniversityCorresponding authors. ###### Abstract While autoregressive Large Vision-Language Models (LVLMs) demonstrate remarkable proficiency in multimodal tasks, they face a “Visual Signal Dilution” phenomenon, where the accumulation of textual history expands the attention partition function, causing visual attention to decay inversely with generated sequence length. To counteract this, we propose Persistent Visual Memory (PVM), a lightweight learnable module designed to ensure sustained, on-demand visual perception. Integrated as a parallel branch alongside the Feed-Forward Network (FFN) in LVLMs, PVM establishes a distance-agnostic retrieval pathway that directly provides visual embeddings for precise visual perception, thereby structurally mitigating the signal suppression inherent to deep generation. Extensive experiments on Qwen3-VL models demonstrate that PVM brings notable improvements with negligible parameter overhead, delivering consistent average accuracy gains across both 4B and 8B scales, particularly in complex reasoning tasks that demand persistent visual perception. Furthermore, in-depth analysis reveals that PVM can resist length-induced signal decay and accelerate internal prediction convergence. Our code is available at . 1 Introduction -------------- The rapid evolution of Large Vision-Language Models (LVLMs)*[[15](#bib.bib35 "Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities"), [3](#bib.bib77 "Llava-onevision-1.5: fully open framework for democratized multimodal training"), [5](#bib.bib1 "Qwen3-vl technical report"), [33](#bib.bib78 "GLM-4.1 v-thinking: towards versatile multimodal reasoning with scalable reinforcement learning"), [66](#bib.bib79 "Kimi-vl technical report"), [78](#bib.bib43 "Internvl3. 5: advancing open-source multimodal models in versatility, reasoning, and efficiency")]* has established a dominant paradigm in multimodal intelligence. By bridging vision encoders with Large Language Models (LLMs) via semantic projection, models like the Qwen3-VL*[[5](#bib.bib1 "Qwen3-vl technical report")]* series have achieved remarkable proficiency across diverse tasks, ranging from fine-grained visual perception to complex reasoning*[[40](#bib.bib80 "A survey on multimodal benchmarks: in the era of large ai models"), [64](#bib.bib81 "Thinking with images for multimodal reasoning: foundations, methods, and future frontiers")]*. This modeling approach allows visual tokens to be processed seamlessly alongside textual sequences, effectively extending the cognitive scope of LLMs to encompass visual perception. As these capabilities mature, the research frontier is increasingly shifting towards deep generation scenarios, where models are expected to sustain high-level understanding and interaction across extended dialogues and multi-step deductive chains*[[18](#bib.bib82 "Insight-v: exploring long-chain visual reasoning with multimodal large language models"), [65](#bib.bib83 "Mitigating visual forgetting via take-along visual conditioning for multi-modal long cot reasoning"), [63](#bib.bib84 "Long-vita: scaling large multi-modal models to 1 million tokens with leading short-context accuracy")]*. However, this expansion into extended contexts faces a critical Visual Signal Dilution. In the standard autoregressive framework, visual tokens serve as fixed precursors*[[46](#bib.bib37 "Visual instruction tuning"), [5](#bib.bib1 "Qwen3-vl technical report"), [45](#bib.bib106 "Improved baselines with visual instruction tuning"), [85](#bib.bib107 "Autoregressive models in vision: a survey")]*; unlike text which naturally self-replenishes during generation, they cannot be intrinsically regenerated. Our analysis reveals that the attention mechanism inherently creates a structural conflict: as textual history accumulates, the normalization induced by attention over an ever-growing context redistributes probability mass across more tokens*[[71](#bib.bib108 "Long-context generalization with sparse attention"), [51](#bib.bib109 "Sieve attention: fusing context-aware filtering and sequential allocation for long sequence")]*, causing the once-injected visual signals to be progressively attenuated. This process drives the model through a phase of asymptotic decay into a Low-Attention Equilibrium, where visual cues are outweighed by textual priors by orders of magnitude*[[41](#bib.bib113 "Evaluating object hallucination in large vision-language models"), [49](#bib.bib114 "Vision-language introspection: mitigating overconfident hallucinations in mllms via interpretable bi-causal steering"), [35](#bib.bib115 "Enhancing visual reliance in text generation: a bayesian perspective on mitigating hallucination in large vision-language models")]*. Consequently, maintaining fidelity requires a shift from passive retention to sustained, on-demand perception, enabling the model to dynamically re-examine visual memory*[[96](#bib.bib110 "Learning without forgetting for vision-language models"), [52](#bib.bib111 "Seeing, listening, remembering, and reasoning: a multimodal agent with long-term memory"), [47](#bib.bib112 "Memverse: multimodal memory for lifelong learning agents")]*. Refer to caption *Figure 1: Visual Memory Mechanisms. Unlike Standard LVLMs that degrade via visual dilution or Injection methods that cause serial interference, our Persistent Visual Memory (PVM) establishes an independent retrieval path, preserving visual intensity without disrupting the autoregressive flow.* To counteract this decay, prior works have attempted to explicitly re-inject visual information, a paradigm we characterize as Visual Injection in Figure[1](#S1.F1 "Figure 1 ‣ 1 Introduction ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"). Whether by inserting raw tokens*[[23](#bib.bib87 "Interleaved-modal chain-of-thought"), [74](#bib.bib89 "Vgr: visual grounded reasoning")]* or fusing processed features*[[89](#bib.bib59 "Vismem: latent vision memory unlocks potential of vision-language models"), [84](#bib.bib90 "Boosting reasoning in large multimodal models via activation replay")]*, these approaches typically inject visual retrieval signals directly into the serial autoregressive pathway. While effective for short-term recall, they can disrupt linguistic coherence. By perturbing the evolving semantic states used for step-by-step reasoning, such interventions create a trade-off: reinforcing visual presence comes at the cost of disturbing the precise logical states required for complex deduction*[[75](#bib.bib91 "VPTracker: global vision-language tracking via visual prompt and mllm"), [67](#bib.bib92 "Large language models reasoning abilities under non-ideal conditions after rl-fine-tuning")]*. We bridge this gap by proposing Persistent Visual Memory (PVM), a lightweight learnable module designed to provide sustained visual perception within the autoregressive flow (as shown in Figure[1](#S1.F1 "Figure 1 ‣ 1 Introduction ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs")). Structurally, PVM is integrated alongside the Feed-Forward Network (FFN) in the Transformer block, effectively bifurcating the generation flow: while the original FFN preserves the model’s reasoning logic, the parallel PVM branch serves as a dedicated channel for retrieving raw visual evidence*[[98](#bib.bib12 "Look twice before you answer: memory-space visual retracing for hallucination mitigation in multimodal large language models"), [25](#bib.bib13 "Transformer feed-forward layers are key-value memories")]*. By employing a gated cross-attention mechanism that attends exclusively to the original visual embeddings, PVM establishes a distance-agnostic retrieval pathway. This design structurally mitigates visual signal suppression from the growing textual history, helping the model maintain access to high-fidelity image details throughout extended contexts. To validate the effectiveness of our proposed PVM, we conduct extensive experiments across a suite of eight challenging multimodal benchmarks, covering general understanding and scientific reasoning. Based on the Qwen3-VL series, our 8B variant with PVM achieves a remarkable 4.8% average accuracy improvement over the baseline, while demonstrating superior performance among open-source models. Notably, our PVM also delivers a consistent 4.4% improvement when applied to the 4B architecture. Furthermore, PVM exhibits explicit resilience against signal decay in extended generation, and mechanistic analysis via LogitLens*[[58](#bib.bib96 "Interpreting gpt: the logit lens"), [14](#bib.bib95 "Conditional memory via scalable lookup: a new axis of sparsity for large language models")]* verifies it accelerates internal prediction convergence rather than merely increasing model capacity. To sum up, our main contributions are threefold: * • We provide a rigorous analysis of Visual Signal Dilution, mathematically deriving the asymptotic decay of visual attention and uncovering the structural conflict that reveals the necessity for a sustained perception paradigm. * • We propose PVM, a lightweight parallel branch that enables sustained, distance-agnostic visual retrieval. We theoretically characterize how this architecture structurally mitigates visual signal suppression and preserves exact visual grounding. * • We achieve substantial gains across eight benchmarks, demonstrating robust scalability on both 4B (+4.4%) and 8B (+4.8%) backbones. Further analyses confirm PVM resists length-induced signal decay and accelerates internal prediction convergence. 2 Related Work -------------- ##### General LVLMs and Challenges in Visual Persistence. The rapid evolution of Large Language Models (LLMs)*[[9](#bib.bib33 "Language models are few-shot learners"), [1](#bib.bib32 "Gpt-4 technical report"), [26](#bib.bib34 "The llama 3 herd of models"), [86](#bib.bib3 "Qwen3 technical report")]* has catalyzed the development of Large Vision-Language Models (LVLMs). By bridging pre-trained visual encoders*[[60](#bib.bib41 "Learning transferable visual models from natural language supervision"), [69](#bib.bib2 "Siglip 2: multilingual vision-language encoders with improved semantic understanding, localization, and dense features")]* with LLMs via learnable interfaces, ranging from linear layers*[[13](#bib.bib36 "Internvl: scaling up vision foundation models and aligning for generic visual-linguistic tasks"), [46](#bib.bib37 "Visual instruction tuning"), [97](#bib.bib40 "Minigpt-4: enhancing vision-language understanding with advanced large language models")]* and Q-Formers*[[39](#bib.bib42 "Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models"), [4](#bib.bib38 "Qwen-vl: a frontier large vision-language model with versatile abilities"), [17](#bib.bib39 "Instructblip: towards general-purpose vision-language models with instruction tuning")]* to more advanced variants*[[78](#bib.bib43 "Internvl3. 5: advancing open-source multimodal models in versatility, reasoning, and efficiency"), [5](#bib.bib1 "Qwen3-vl technical report")]*, these models have achieved remarkable proficiency across diverse tasks, including visual perception*[[21](#bib.bib45 "MME: a comprehensive evaluation benchmark for multimodal large language models"), [50](#bib.bib23 "Mmbench: is your multi-modal model an all-around player?")]*, complex reasoning*[[59](#bib.bib19 "We-math 2.0: a versatile mathbook system for incentivizing visual mathematical reasoning"), [95](#bib.bib26 "Mathverse: does your multi-modal llm truly see the diagrams in visual math problems?"), [36](#bib.bib102 "Spotlight on token perception for multimodal reinforcement learning"), [29](#bib.bib105 "Framethinker: learning to think with long videos via multi-turn frame spotlighting"), [80](#bib.bib116 "Sampling-efficient test-time scaling: self-estimating the best-of-n sampling in early decoding"), [31](#bib.bib117 "Diffthinker: towards generative multimodal reasoning with diffusion models"), [30](#bib.bib120 "VideoSSR: video self-supervised reinforcement learning")]*, and open-ended dialogue*[[77](#bib.bib74 "Characterbox: evaluating the role-playing capabilities of llms in text-based virtual worlds"), [2](#bib.bib73 "Pixtral 12b")]*. However, despite these advancements, ensuring faithfulness in deep generation remains a formidable challenge; as generation extends into longer responses, LVLMs exhibit a pronounced susceptibility to hallucinations*[[6](#bib.bib85 "Hallucination of multimodal large language models: a survey"), [44](#bib.bib86 "A survey on hallucination in large vision-language models")]*, where generated content diverges from visual facts. Rather than being a mere training artifact, this degradation is increasingly recognized as a fundamental architectural bottleneck: as the autoregressive history grows, the static visual tokens located at the sequence start are statistically drowned out by the expanding textual priors*[[41](#bib.bib113 "Evaluating object hallucination in large vision-language models"), [49](#bib.bib114 "Vision-language introspection: mitigating overconfident hallucinations in mllms via interpretable bi-causal steering"), [35](#bib.bib115 "Enhancing visual reliance in text generation: a bayesian perspective on mitigating hallucination in large vision-language models")]*, creating an urgent need for mechanisms that can sustain visual perception over long horizons. ##### Visual Injection and Context Management. Addressing the volatility of visual signals in extended contexts, recent research has diverged into three primary streams. Hierarchical systems and retrieval-augmented frameworks seek to decouple storage from processing through dual-stream architectures or recurrent bridges*[[89](#bib.bib59 "Vismem: latent vision memory unlocks potential of vision-language models"), [27](#bib.bib60 "ContextualLVLM-agent: a holistic framework for multi-turn visually-grounded dialogue and complex instruction following"), [54](#bib.bib61 "ViDove: a translation agent system with multimodal context and memory-augmented reasoning"), [48](#bib.bib62 "CoMemo: lvlms need image context with image memory"), [81](#bib.bib63 "Videollamb: long-context video understanding with recurrent memory bridges"), [7](#bib.bib64 "Memory consolidation enables long-context video understanding"), [90](#bib.bib65 "Memory-enhanced retrieval augmentation for long video understanding"), [22](#bib.bib118 "LatentMem: customizing latent memory for multi-agent systems"), [28](#bib.bib119 "GEMS: agent-native multimodal generation with memory and skills")]*. To manage computational constraints, visual token compression and bottleneck mechanisms reduce feature granularity via resampling*[[10](#bib.bib69 "Fwd2Bot: lvlm visual token compression with double forward bottleneck"), [20](#bib.bib70 "Vision remember: alleviating visual forgetting in efficient mllm with vision feature resample"), [62](#bib.bib71 "Context-aware multi-turn visual-textual reasoning in lvlms via dynamic memory and adaptive visual guidance")]*, often extended to continual learning scenarios*[[68](#bib.bib72 "Learning long-context diffusion policies via past-token prediction")]*. Additionally, attention optimization strategies employ sparse matrices and decay resilience techniques to mitigate long-range hallucination*[[53](#bib.bib66 "MDSAM: memory-driven sparse attention matrix for lvlms hallucination mitigation"), [24](#bib.bib67 "Remember me: bridging the long-range gap in lvlms with three-step inference-only decay resilience strategies"), [83](#bib.bib68 "ChainMPQ: interleaved text-image reasoning chains for mitigating relation hallucinations")]*. However, these approaches often necessitate complex external modules, structurally alter the backbone, or inevitably sacrifice fine-grained visual fidelity. In contrast, we propose PVM, a parallel retrieval mechanism that preserves original visual features without disrupting the autoregressive flow. 3 Analysis of Visual Signal Dilution ------------------------------------ In this section, we analyze the architectural bottleneck in autoregressive LVLMs. We provide a theoretical framework describing how standard attention inherently dilutes visual signals in Section[3.1](#S3.SS1 "3.1 Theoretical Formulation ‣ 3 Analysis of Visual Signal Dilution ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"), and empirically validate these dynamics to offer insights for our method design in Section[3.2](#S3.SS2 "3.2 Empirical Verification ‣ 3 Analysis of Visual Signal Dilution ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"). ### 3.1 Theoretical Formulation Consider an autoregressive LVLM generating a new token. The input context is partitioned into a fixed set of visual tokens $\mathcal{V}$ ($|\mathcal{V}|\=M$) and a dynamically growing set of textual history tokens $\mathcal{T}_{t}$ ($|\mathcal{T}_{t}|\=t$). For the current query vector $\mathbf{q}_{t}$ in a Transformer block*[[70](#bib.bib14 "Attention is all you need")]*, let $s_{k}(\mathbf{q}_{t})\=(\mathbf{q}_{t}^{\top}\mathbf{k}_{k})/\sqrt{d}$ be the unnormalized attention score assigned to the $k$-th context token, where $\mathbf{k}_{k}$ is the key vector and $d$ is the head dimension. To dissect the attention mechanism, we decompose the partition function (the Softmax denominator) into the aggregate unnormalized mass of the visual context $Z_{\mathcal{V}}$ and the textual history $Z_{\mathcal{T}}$: $Z_{\mathcal{V}}(\mathbf{q}_{t})\=\sum_{k\in\mathcal{V}}\exp(s_{k}(\mathbf{q}_{t})),\,Z_{\mathcal{T}}(\mathbf{q}_{t},t)\=\sum_{k\in\mathcal{T}_{t}}\exp(s_{k}(\mathbf{q}_{t}))$. We explicitly parameterize $Z_{\mathcal{T}}$ with $t$ to denote the underlying cardinality growth of the summation over time. In standard self-attention, the probability weight $\alpha_{t,k}$ for the $k$-th token is given by $\exp(s_{k}(\mathbf{q}_{t}))/(Z_{\mathcal{V}}(\mathbf{q}_{t})+Z_{\mathcal{T}}(\mathbf{q}_{t},t))$. We define the Visual Attention Mass, $\Omega_{\mathcal{V}}(t)$, as the aggregate weight allocated to the visual set: | | $\Omega_{\mathcal{V}}(t):\=\sum_{k\in\mathcal{V}}\alpha_{t,k}\=\frac{\sum_{k\in\mathcal{V}}\exp(s_{k}(\mathbf{q}_{t}))}{Z_{\mathcal{V}}(\mathbf{q}_{t})+Z_{\mathcal{T}}(\mathbf{q}_{t},t)}\=\frac{Z_{\mathcal{V}}(\mathbf{q}_{t})}{Z_{\mathcal{V}}(\mathbf{q}_{t})+Z_{\mathcal{T}}(\mathbf{q}_{t},t)}$ | | (1) | | --- | --- | --- | --- | While $Z_{\mathcal{V}}(\mathbf{q}_{t})$ is bounded by the fixed number of visual tokens $M$, $Z_{\mathcal{T}}(\mathbf{q}_{t},t)$ aggregates over a growing textual history $t$. We characterize the asymptotic behavior of $\Omega_{\mathcal{V}}(t)$ through two distinct operational phases: the Power-Law Dilution Phase and the High-Magnitude Saturation Phase. #### 3.1.1 Phase I: Power-Law Dilution via Active Competition In the early-to-mid stage of generation, the model must continuously reference historical context to maintain coherence. We posit a Persistent textual attention condition: historical text tokens maintain a statistically significant relevance, bounded below by a positive constant $\mu>0$ such that the average unnormalized attention mass satisfies $t^{-1}Z_{\mathcal{T}}(\mathbf{q}_{t},t)\geq\mu$. However, unlike the growing textual history, the visual information is constrained by the fixed number of visual tokens $M$. In practice, input normalization constrains the pre-softmax scores, effectively imposing an upper bound $s_{\max}$. Consequently, the aggregate visual mass $Z_{\mathcal{V}}(\mathbf{q}_{t})$ is upper-bounded by $\beta\=M\cdot\exp(s_{\max})$. Based on this structural conflict, we formulate the dilution theorem: ###### Theorem 3.1(Visual Signal Dilution). Given fixed visual context size $M$ and textual attention lower bound $\mu$, the visual attention mass $\Omega_{\mathcal{V}}(t)$ is structurally dominated by the growing textual history: | | $\Omega_{\mathcal{V}}(t)\leq\frac{\beta}{\beta+\mu\cdot t}\=\mathcal{O}(t^{-1})$ | | (2) | | --- | --- | --- | --- | ###### Proof. Substituting the visual upper bound $Z_{\mathcal{V}}\leq\beta$ and the textual lower bound $Z_{\mathcal{T}}\geq\mu t$ into Eq.[1](#S3.E1 "In 3.1 Theoretical Formulation ‣ 3 Analysis of Visual Signal Dilution ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs") yields the governing inequality: $\Omega_{\mathcal{V}}(t)\=\frac{Z_{\mathcal{V}}}{Z_{\mathcal{V}}+Z_{\mathcal{T}}}\leq\frac{\beta}{\beta+Z_{\mathcal{T}}}\leq\frac{\beta}{\beta+\mu\cdot t}$. As $t$ grows, the linear term $\mu\cdot t$ in the denominator dominates the constant $\beta$, implying $\Omega_{\mathcal{V}}(t)\=\mathcal{O}(t^{-1})$. ∎ #### 3.1.2 Phase II: The High-Magnitude Saturation Trap In practice, $Z_{\mathcal{T}}$ eventually saturates due to effective attention limits. However, we argue that this saturation does not rescue the visual signal; rather, it locks the model into a Low-Attention Equilibrium. Let $W_{eff}$ denote the effective textual window size. Since the effective textual window is typically orders of magnitude larger than the visual context ($W_{eff}\gg M$), the aggregate textual mass stabilizes at a high plateau $Z_{\mathcal{T}}^{sat}$. Thus, the system converges to: $\lim_{t\to\infty}\Omega_{\mathcal{V}}(t)\approx\frac{\mathbb{E}[Z_{\mathcal{V}}]}{Z_{\mathcal{T}}^{sat}}\ll 1$. Although the dilution halts, the system stabilizes with a heavily skewed distribution. The visual signal remains structurally outweighed by the overwhelming textual momentum, making the model more prone to errors when precise visual grounding is required. ### 3.2 Empirical Verification To validate our theory, we analyze attention dynamics using Qwen3-VL-8B-Instruct*[[5](#bib.bib1 "Qwen3-vl technical report")]* on a subset of the COCO 2017*[[43](#bib.bib5 "Microsoft coco: common objects in context")]* validation set. We designed a “Blind Painter” Stress Test (see Appendix[J](#A10 "Appendix J Prompt Templates ‣ 7 Conclusion ‣ Extended Analyses. ‣ 6.4 Ablation Studies ‣ 6.3 Mechanistic Analysis of PVM ‣ 6.2 Robustness to Extended Generation ‣ 6.1 Main Results ‣ 6 Results ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs")). Specifically, we prompt the model to reconstruct the image textually with extreme detail, as if instructing a painter who cannot see the original work. This setup necessitates active visual retrieval at every generation step rather than just the beginning, creating a rigorous testbed that isolates signal dilution from general language generation. Refer to caption *Figure 2: Power-Law Decay of Visual Signal.Log-scale analysis confirms that $\Omega_{\mathcal{V}}$ tightly follows the $\mathcal{O}(t^{-1})$ trajectory predicted by Theorem[3.1](#S3.Thmtheorem1 "Theorem 3.1 (Visual Signal Dilution). ‣ 3.1.1 Phase I: Power-Law Dilution via Active Competition ‣ 3.1 Theoretical Formulation ‣ 3 Analysis of Visual Signal Dilution ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"). This demonstrates that visual attention mass is structurally diluted, decaying inversely to the sequence length $t$.* Refer to caption *Figure 3: Evolution of Textual Dominance.The TVR trajectory effectively validates the proposed two-phase dilution mechanism. This reveals how initial probability competition saturates into a strict equilibrium, where textual priors structurally overwhelm visual signals.* #### 3.2.1 The Phenomenon: Spatiotemporal Decay of Visual Mass We track the Visual Attention Mass $\Omega_{\mathcal{V}}$ throughout the generation process. As shown in the logarithmic analysis (Figure[3](#S3.F3 "Figure 3 ‣ 3.2 Empirical Verification ‣ 3 Analysis of Visual Signal Dilution ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs")), the empirical decay trajectory closely aligns with the $\mathcal{O}(t^{-1})$ power-law predicted by Theorem[3.1](#S3.Thmtheorem1 "Theorem 3.1 (Visual Signal Dilution). ‣ 3.1.1 Phase I: Power-Law Dilution via Active Competition ‣ 3.1 Theoretical Formulation ‣ 3 Analysis of Visual Signal Dilution ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"). Crucially, layer-wise analysis reveals that this attenuation is pronounced in the intermediate layers (Layers 8–27), which serve as the primary locus for multimodal reasoning (detailed in Appendix[A](#A1 "Appendix A Visual Attention Mass Heatmap Analysis ‣ 7 Conclusion ‣ Extended Analyses. ‣ 6.4 Ablation Studies ‣ 6.3 Mechanistic Analysis of PVM ‣ 6.2 Robustness to Extended Generation ‣ 6.1 Main Results ‣ 6 Results ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs")). Thus, even though these layers retain the highest relative visual relevance, they are forced to operate on a progressively weakening visual foundation as the text expands, undermining the model’s ability to sustain precise grounding. #### 3.2.2 The Mechanism: Dynamics of Textual Dominance To elucidate the mathematical driver of attenuation, we analyze the Text-to-Vision Ratio (TVR): $\text{TVR}(t)\=Z_{\mathcal{T}}(\mathbf{q}_{t},t)/Z_{\mathcal{V}}(\mathbf{q}_{t})$. Figure[3](#S3.F3 "Figure 3 ‣ 3.2 Empirical Verification ‣ 3 Analysis of Visual Signal Dilution ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs") validates the two-phase dynamics: the trajectory initially exhibits linear growth (Phase I), confirming that historical text acts as an active competitor for probability mass. Crucially, it eventually stabilizes at a saturation plateau (Phase II) where the textual mass exceeds the visual mass by nearly two orders of magnitude. This quantifies the Low-Attention Equilibrium: at such high ratios, the visual signal $Z_{\mathcal{V}}$ is statistically insignificant in the shared partition function, limiting the model’s ability to maintain strict visual fidelity in deep generation. 4 Method: Persistent Visual Memory ---------------------------------- Guided by the theoretical analysis and insights in Section[3](#S3 "3 Analysis of Visual Signal Dilution ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"), we propose Persistent Visual Memory (PVM). As shown in Figure[4](#S4.F4 "Figure 4 ‣ Gated Fusion with Selective Activation. ‣ 4.1 Architecture Design ‣ 4 Method: Persistent Visual Memory ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"), this framework operationalizes the insights derived from the dilution bottleneck: it satisfies the need for a parallel memory pathway and enforces independent attention normalization. In Section[4.1](#S4.SS1 "4.1 Architecture Design ‣ 4 Method: Persistent Visual Memory ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"), we detail the parallel architecture and its gated fusion mechanism. Subsequently, in Section[4.2](#S4.SS2 "4.2 Theoretical Guarantee ‣ 4 Method: Persistent Visual Memory ‣ Persistent Visual Memory: Sustaining Perception for Deep Generation in LVLMs"), we provide a theoretical analysis showing that PVM structurally mitigates the impact of growing textual context on visual retrieval. ### 4.1 Architecture Design Unlike previous approaches relying on external retrieval*[[87](#bib.bib9 "Retrieval-augmented multimodal language modeling"), [11](#bib.bib10 "Wiki-llava: hierarchical retrieval-augmented generation for multimodal llms"), [55](#bib.bib11 "A survey of multimodal retrieval-augmented generation")]*, we integrate PVM into the Transformer decoder block as a parallel branch. Specifically, inspired by MemVR*[[98](#bib.bib12 "Look twice before you answer: memory-space visual retracing for hallucination mitigation in multimodal large language models")]* and the understanding of Feed-Forward Networks as key-value memories*[[25](#bib.bib13 "Transformer feed-forward layers are key-value memories")]*, we position the PVM parallel to the FFN sub-layer to function as an active perception channel for the current visual context. Let $\mathbf{x}\in\mathbb{R}^{d}$ denote the hidden state output from the Multi-Head Self-Attention (MHSA) layer. Departing from the standard serial architecture where $\mathbf{x}$ is fed exclusively into the FFN, our design bifurcates the information flow into two parallel streams: * • The Reasoning Path (Original): The hidden state travels through the frozen FFN to access pre-trained static knowledge and logical patterns: $\mathbf{h}_{\mathrm{ffn}}\=\mathrm{FFN}(\mathbf{x})$. * • The Looking Path (PVM): Simultaneously, the same state $\mathbf{x}$ acts as the Query for the PVM module. This establishes the parallel memory pathway that is structurally isolated from the autoregressive bottleneck, allowing the model to actively retrieve visual details. ##### PVM Computation. To ensure parameter efficiency and minimize inference overhead, PVM serves as a bottleneck adapter in a projected latent space rather than the full model dimension $d$. Let $\mathbf{x}\in\mathbb{R}^{d}$ be the input hidden state and $\mathbf{V}_{\mathrm{img}}\in\mathbb{R}^{M\times d}$ represent the features of the fixed visual set $\mathcal{V}$. The computation proceeds in three stages: 1. 1. Projection: We project the input hidden state and visual features into a lower-dimensional latent space $d^{\prime}