概念

DeepSeek R1 Distillation refers to the distillation approach used in the DeepSeek R1 model family, where reasoning capabilities are transferred from a large teacher model to smaller student models. This approach is discussed in the on-policy-distillation-survey as a prominent exa...

将差分隐私（Differential Privacy）与 OPD 结合，在保持蒸馏效果的同时提供正式的隐私保证。

|# Dual Policy Distillation (DPD)

|# Entropy-Aware On-Policy Distillation (EOPD)

Ex-OPD (Extended/Extrapolated On-Policy Distillation) is a variant of on-policy distillation that extends the OPD framework with reward extrapolation capabilities. Referenced in the context of generalized-on-policy-distillation, Ex-OPD explores how on-policy methods can be augmen...

*DeepMind's foundational work on on-policy distillation (2023)*

|# Generalized On-Policy Distillation (G-OPD)

Hybrid Distillation Policy Optimization via Privileged Self-Distillation (HDPO) augments standard RL training with a privileged self-distillation mechanism targeting "cliff prompts" — prompts where all student rollouts fail. The teacher and student share weights and differ only i...

混合策略蒸馏方法，结合多种蒸馏策略以平衡不同训练目标。仅有 abstract 可用，全文尚未公开。

KL (Kullback-Leibler) divergence plays a central role in knowledge distillation as the primary loss function measuring the discrepancy between teacher and student output distributions. The on-policy-distillation-survey frames distillation methods through the lens of f-divergence ...

Knowledge Distillation (KD), introduced by Hinton et al. in 2015, is the foundational technique for transferring knowledge from a large "teacher" model to a smaller "student" model. The student is trained to match the teacher's output distribution (typically via kl-divergence-in-...

NVIDIA 团队（Song Han, Han Cai）提出的离线 OPD 变体，通过预计算 teacher log-probabilities 消除实时推理开销，实现 4 倍加速。

|# Model Distillation

Multi-domain on-policy distillation extends the OPD framework to handle multiple task domains simultaneously, training a single student model that can perform across diverse domains by distilling from domain-specific or generalist teachers. This concept is related to cascade-rl, ...

|# On-Policy Distillation (OPD)

|# On-Policy Distillation (OPD) - Survey

|# On-Policy Prefix Distillation (OPPD)

|# On-Policy Self-Distillation (OPSD)

On-Policy Self-Distillation for Reasoning Compression (OPSDC) is a training method that compresses LLM reasoning traces by conditioning the same model on a "be concise" instruction to generate teacher logits, then training the student (unconditioned) to minimize per-token reverse...

将 OPD 扩展到 long-context 场景，通过自蒸馏扩展 LLM 有效上下文长度。

|# Per-Token Pointwise KL Clipping

|# Policy Distillation

|# Proximal Policy Distillation (PPD)

Reasoning distillation is the process of transferring reasoning abilities — particularly chain-of-thought capabilities — from a teacher model to a student model. This is a specialized form of knowledge-distillation that focuses on preserving the quality and structure of reasoning...

Reinforcement-Aware Knowledge Distillation (RLAD) is an on-policy distillation framework that selectively imitates the teacher during RL training — distilling from the teacher only when it improves the current policy update. RLAD replaces the standard KL regularizer with a Trust ...

|# Reopold (Relaxed On-Policy Distillation)

清华/上海交大等机构对 OPD 动力学的系统性研究，揭示成功 OPD 的必要条件、token 级别机制，以及恢复失败 OPD 的策略。

Reverse KL distillation uses the reverse Kullback-Leibler divergence (KL(student || teacher)) instead of the forward direction as the distillation objective. Unlike forward KL, which encourages the student to cover all modes of the teacher distribution, reverse KL encourages the ...

Sample-Routed Policy Optimization (SRPO) is a unified on-policy post-training framework that dynamically routes correct samples to GRPO-style reward-aligned reinforcement and failed samples to SDPO's (Self-Distillation Policy Optimization) targeted logit-level correction. It uses...

通过自修订（self-revision）机制将 binary rewards 转化为 dense supervision，提供对比 RLVR 与 self-distillation 的视角。

将 OPD 与 RLVR（Reinforcement Learning with Verifiable Rewards）结合，用大模型 teacher 为采样轨迹提供密集细粒度信号。

Strong-to-weak distillation is a paradigm where a larger, more capable "strong" teacher model distills its knowledge into a smaller, less capable "weak" student model. This is the most common distillation direction and is used in models like qwen3 and frameworks like generalized-...

Teacher top-k local support matching is a technique used in knowledge distillation where the student is trained to match not just the full output distribution of the teacher, but specifically the top-k highest-probability tokens at each position. This provides more focused traini...

Salesforce AI Research 发现 OPD 会导致严重过置信（overconfidence），提出 CaOPD 解耦能力与校准。

研究 token 级别重要性在 OPD 中的作用，发现并非所有 token 位置对蒸馏效果贡献相同。

Token-level entropy analysis is a diagnostic technique for evaluating how well a student model preserves the uncertainty structure of a teacher model during knowledge distillation. It involves computing the Shannon entropy of the conditional output distribution at each token posi...

TVDF (Temporal Video Distillation Framework) is a distillation framework focused on video understanding tasks, where temporal information plays a critical role. TVDF is referenced in the context of video-opd, which applies on-policy distillation techniques to video temporal groun...

Video-OPD (Video On-Policy Distillation) is an efficient post-training framework for Temporal Video Grounding (TVG) that combines on-policy optimization with dense token-level supervision from a fixed frontier teacher model.

VOLD is a two-stage framework for transferring reasoning capabilities from text-only teacher LLMs to vision-language student models (VLMs) using purely text-based training data. It combines Group Relative Policy Optimization (GRPO) reinforcement learning with on-policy knowledge ...

X-OPD is a novel Cross-Modal On-Policy Distillation framework designed to systematically align the capabilities of Speech LLMs to their text-based counterparts. It enables the Speech LLM to explore its own distribution via on-policy rollouts, where a text-based teacher model eval...

*Framework for training powerful LLM agents with reinforcement learning (2025)*

LLM Agent 能否自主设计、实现、运行完整的 RL pipeline 来改进 foundation model？

|# Cascade RL

LLM 在长程交互环境中做 consistent 决策困难，因为缺乏发现、保留和复用结构化 skill 的机制。现有 LLM agent 没有类似 Atari/AlphaZero 中 self-improving game agent 的 explore-learn-reuse 循环。

Deep Q-Network (DQN) is a deep reinforcement learning value-based method that combines Q-learning with deep neural networks to learn policies directly from high-dimensional sensory input. Introduced by DeepMind in 2015, DQN achieved human-level performance on atari-2600 games and...

RLVR 训练的 VLM 在视觉空间推理上存在 accuracy-faithfulness 矛盾：

GRPO 在多模态多任务 RL 中存在 reward topology 方差极大 的问题：

本文系统性研究了一个在现有 RL 训练中常被忽视的因素——horizon length（视野长度 / 任务执行步数）如何影响 LLM Agent 训练的稳定性和最终性能。

Imitation Learning (IL) is a paradigm where an agent learns to perform tasks by observing and replicating demonstrations from an expert, rather than learning through trial-and-error with explicit reward signals. It is closely related to policy-distillation in RL settings and serv...

Interactive Imitation Learning extends standard imitation-learning by allowing the learner to query the expert interactively during training, rather than learning only from a fixed dataset of demonstrations. This addresses the exposure-bias problem by ensuring the learner receive...

*Learning-based agentic ML (2025)*

VLM 的 RL 训练中，不是所有 token 都平等——视觉感知相关的 token 应该获得不同的优化策略。

PerlAD 提出了一种基于离线数据集的 伪仿真 (Pseudo-simulation) 环境，在向量空间中实现高效、无需渲染的 RL 训练，用于闭环端到端自动驾驶，在 Bench2Drive 上达到 SoTA (DS=78.70)。

LLM RL 训练中引入双模式探索策略，分离 exploit 和 explore 的优化路径。

|# Proximal Policy Optimization (PPO)

*StarPO framework for trajectory-level agent learning (2025)*

Reinforcement learning (RL) post-training has become essential for unlocking reasoning, self-reflection, and tool-use in large language models. However, as models extend to omni-modal inputs (image, video, audio, text) and agentic multi-turn workflows, existing RL training system...

*Qwen2.5 mathematical reasoning study (2025)*

Formal Statement. Let $\pi_\theta$ be a policy model being pretrained. Standard pretraining optimizes next-token prediction: $\max_\theta \sum_{t} \log \pi_\theta(x_t | x_{<t})$. This ignores desirable properties (safety, factuality, quality) that are only addressed during post-t...

Self-play limitations refer to the constraints and failure modes of self-play approaches in reinforcement learning and model distillation. While self-play has been highly successful in games (e.g., AlphaGo), its application to LLM post-training and distillation faces challenges i...

Stable Baselines3 is a set of reliable implementations of reinforcement learning algorithms in PyTorch. It is the framework upon which the sb3-distill library (implementing proximal-policy-distillation) is built.

在自然语言指令上用 RL 训练，激发 LLM 的通用推理能力——不需要专门的数学/代码推理数据。

RLVR 依赖可验证的 reward signal，在专业/新领域不可用。

视觉引导的策略优化：让 RL 训练的信号同时考虑视觉证据和语言推理。

Agent safety covers methods for preventing tool-using and autonomous agents from executing harmful, unsafe, or policy-violating behavior across planning, memory, tool use, and multi-step execution.

Agentic research ideas are research directions where agents help formulate hypotheses, design experiments, operate tools, and iterate on findings.

Agentic workflows are manually or automatically designed chains of agent steps, tools, roles, and verification loops.

AgentPoison refers to backdoor or poisoning attacks against agent memory, where malicious content is inserted into persistent state and later influences agent decisions.

ASR benchmarks evaluate automatic speech recognition quality across languages, accents, noise conditions, and domain-specific speech.

Complementary Learning Systems is a theory of memory that separates fast episodic acquisition from slower consolidation into structured long-term knowledge.

Diffusion language models generate or refine discrete text through denoising-style processes instead of purely left-to-right autoregression.

EAGLE-3 is referenced in this wiki as an autoregressive drafter baseline for speculative decoding and diffusion-based acceleration methods.

Generative agents are LLM-driven simulated agents that maintain memory streams, reflect over events, and plan behavior in an environment.

LLM inference optimization covers methods that reduce latency, memory, or cost during model serving, including speculative decoding, KV-cache optimization, routing, and compression.

Mixture of Agents is an ensemble-style pattern where multiple agents or models propose, critique, or refine outputs before aggregation.

Multi-agent coordination covers role assignment, communication, aggregation, conflict handling, and shared-state management among multiple agents.

Omni model evolution tracks the progression from text-centric LLMs to unified models that integrate text, vision, audio, speech, and video.

Prompt injection is an attack where untrusted content attempts to override, redirect, or manipulate an LLM or agent's intended instructions.

Qdrant vector search refers to approximate nearest-neighbor retrieval over embedding vectors using the Qdrant vector database.

Reflexion is an agent learning pattern where the agent writes verbal feedback or self-critique after an attempt and reuses that feedback in later attempts.

ROME and MEMIT are model-editing methods for changing factual knowledge stored in model weights.

Skill orchestration is the process of selecting, sequencing, and verifying agent skills or tools for a task.

Test-time scaling allocates more computation during inference through longer reasoning, search, sampling, verification, or multi-agent deliberation.

Agent Safety via RL (arXiv:2507.08270) 是蚂蚁集团提出的统一安全对齐框架，专门处理 LLM Agent 中的用户发起威胁和工具发起威胁。该方法采用三模态分类 (benign/malicious/sensitive) 代替传统的二元安全判断，并在沙箱 RL 环境中训练 Agent 同时抵御两种威胁源。

第三方 API router 作为应用层代理，对每个 JSON payload 有完全明文访问权限。

Traditional scientific publication compresses a branching, iterative research process into a linear narrative, discarding the majority of what was discovered along the way. This compilation imposes two structural costs:

AgentAlign (arXiv:2505.23020) 提出抽象行为链 (Abstract Behavior Chains) 方法来合成 Agent 安全对齐训练数据。核心创新在于从真实工具使用中提取抽象行为模式，然后用非恶意解读这些行为链的比例来合成良性指令，从而解决 agentic 场景下安全数据稀缺的问题。该方法将 LLM Agent 的安全性从 35.8% 提升到 79.5%。

Coding agents consist of a base LLM M surrounded by a harness H — the collection of model-external, editable components: system prompt, tools, middleware, skills, sub-agent configuration, and long-term memory. The harness mediates how M perceives and acts on its environment.

论文提出 agentic world modeling 的三层递进能力体系，从局部预测到全局仿真再到自我演化。

Modern large-scale recommendation systems are multi-stage pipelines (pre-ranking → ranking → re-ranking), where system-level configuration optimization — integrating outputs from various model heads into final scores — is crucial but extremely challenging.

针对 tool-augmented LLM agent 的间接 prompt 注入攻击，提供运行时安全框架。

The paper argues that all current agentic memory systems implement lookup, not memory. This is a category error with provable consequences:

LLM agents increasingly rely on reusable skills — bundles of instructions, control flow, constraints, and callable operations. However, skills are still represented by text-heavy artifacts (SKILL.md-style documents) whose machine-usable evidence remains embedded in natural langua...

No single LLM is universally optimal across all tasks. Different models are fine-tuned to specialize in particular domains. The challenge is: how to dynamically coordinate a pool of diverse, specialized LLMs to solve complex problems that exceed any individual model's capabilitie...

Large reasoning models (LRMs) like DeepSeek-R1 and OpenAI o1 generate extended chains of thought spanning 12,000-25,000 tokens, yet their integration with retrieval-augmented generation (RAG) is fundamentally misaligned.

Question: Can agent collaboration itself be scaled through recursion?

Thought-Aligner (arXiv:2505.11063) 是复旦大学提出的轻量级插件式思维纠正模块，能够在推理时实时检测并纠正 Agent 的高风险思维链 (Chain-of-Thought)。该方法通过对比学习微调一个小型模型，作为"思维监督器"插入到任意 LLM Agent 的推理过程中，将安全性从约 50% 提升至 90%，且延迟控制在 100ms 以内。

WaltzRL (arXiv:2510.08240) 是 Meta 提出的多 Agent 安全协作框架，通过 conversation agent 和 feedback agent 的联合训练实现安全对齐。核心创新是 Dynamic Improvement Reward (DIR) 机制，使两个 Agent 在对抗性协作中共同进化，将 unsafe 率从 39.0% 降至 4.6%，over-refusal 从 45.3% 降至 9.9%。

现有 agent 记忆系统（mem0, MemoryBank）依赖预定义的存储结构和固定工作流，

LLM 固定上下文窗口无法支持长期多会话的上下文一致性。Agent Memory System 通过

长期对话中 agent 记忆面临两大失败模式：

增量记忆处理（如 mem0）存在 有损抽象 (lossy abstraction) 问题：

RAG 和图记忆系统基于 成对关系，无法捕捉高阶关联（多元素间的联合依赖），

用 Small Language Models (SLMs) 替代大模型做记忆操作，分离在线/离线处理。

LLM 受限于固定上下文窗口，无法在多轮交互中管理超出上下文容量的信息。

LLM Agent 的持久记忆面临两大生产挑战：

记忆提取从 被动转录 (one-shot transcription) 转向 主动决策 (reasoning-driven selective extraction)。

现有 agent 记忆系统依赖预定义启发式（重要性评分、情感标签、事实模板）决定"什么值得记住"，

跨异构任务的 LLM 记忆自进化：记忆系统在不同任务间迁移和进化，而非每个任务从零开始。

现有 agent 记忆方案的两难困境：

现有 agent 记忆框架暴露的操作原语不完整且不一致：

CosyVoice 是阿里 Speech Lab（FunAudioLLM 团队）于 2024 年 7 月发布的大规模多语言零样本 TTS 合成系统。核心创新是首次将有监督语义语音 token（Supervised Semantic Speech Tokens, S³）引入 TTS 模型，相比无监督 token 在内容一致性和说话人相似度上显著提升。

CosyVoice 2 是阿里 FunAudioLLM 团队于 2024 年 12 月发布的可扩展流式语音合成模型（arXiv 2412.10117）。相比 cosyvoice 一代，核心改进集中在四个方面：FSQ 替换 VQ 提升码本利用率、直接用预训练 LLM 作为文本-语音 LM 骨干、统一流式/非流式合成的单模型框架、以及增强的指令控制能力。

CosyVoice 3 是阿里通义语音团队（Speech Team, Tongyi Lab, Alibaba Group）于 2025 年 5 月发布的第三代大规模零样本语音生成模型。相比 CosyVoice 2，CV3 在语言覆盖、数据规模、模型容量、后训练技术和评测基准上全面升级，目标是实现in-the-wild speech generation——在真实、嘈杂、多样化的现实场景中生成高质量语音。

DiffRO 是阿里通义语音团队于 2025 年 7 月提出的一种针对基于神经 codec LLM 的 TTS 系统的新型后训练方法。全称 Differentiable Reward Optimization，核心创新是直接在 codec token 上计算奖励并使奖励函数完全可微，从而绕过传统 RLHF 中昂贵的波形生成步骤。

FunCodec is a fundamental, reproducible, and integrable open-source toolkit for neural speech codecs, built as an extension of the FunASR ecosystem. It provides training recipes and inference scripts for modern codec models (SoundStream, Encodec) and introduces FreqCodec — a freq...

Autoregressive (AR) ASR models (Transformer, Conformer) generate tokens sequentially:

The original Paraformer's CIF predictor works well for Mandarin (character-level tokenization) but faces two critical limitations:

Speaker-Attributed ASR (SA-ASR) must solve two tasks simultaneously:

Commercial ASR systems need hotword customization — users should be able to bias recognition toward names, places, domain-specific terms. Prior approaches:

SenseVoice is a speech foundation model family designed for multi-task voice understanding. It integrates four capabilities into a single model:

|# Speech LLM

DuplexCascade 是一种无需 VAD 的级联流式流水线，用于全双工语音到语音对话。其核心创新是将传统的整句轮次（utterance-wise long turns）转换为分块微轮次（chunk-wise micro-turns）交互，并通过精心设计的对话控制特殊 token 来引导 LLM 在流式约束下的行为，从而在保留文本 LLM 强大推理能力的同时实现全双工交互。

MTR-DuplexBench 是首个面向全双工语音语言模型（FD-SLM）的多轮综合评测基准。它不仅将连续全双工对话分割为离散轮次进行逐轮评估，还涵盖了对话质量、对话特征、指令跟随和安全性四个评价维度，弥补了现有基准仅评估单轮交互、仅关注对话特征的不足。

本文首次系统研究了端到端全双工语音对话模型中 LLM 隐藏状态的说话人身份泄露问题，并提出了两种基于 Stream-Voice-Anon 的流式匿名化方案来缓解这一隐私风险。

FLAIR（Full-duplex LAtent and Internal Reasoning）是一种面向全双工口语对话语言模型（SDLM）的潜在推理方法。其核心思想是：在用户说话阶段（listening phase），模型不再预测无意义的静音 token，而是通过递归地将上一步的潜在嵌入作为下一步输入，实现连续的隐式推理（"边听边想"），且不引入额外推理延迟。

TurnGuide 是一种面向端到端（e2e）全双工语音语言模型（FD-SLM）的文本-语音交织生成方法。其核心思想是动态地将助手语音分割为对话轮次（turns），并在每个轮次内交替生成文本和语音，使 FD-SLM 在不破坏自然声学流的前提下，利用 LLM 的文本语义能力来提升对话质量。

全双工 Agent 融合架构是指将全双工对话能力（同时听和说、打断、判停）与 Agent 能力（Function Call、Tool 使用）结合的系统架构。核心挑战在于：轮次管理和 Function Call 执行可能冲突。

全双工语音模型是指能够在同一时刻同时"听"和"说"的语音交互系统。区别于传统的半双工（half-duplex）"你讲我听、我讲你听"轮流机制，全双工模型实现了真正意义上的实时双向交互，更接近人类自然对话方式。

Most unified understanding-generation models are built on autoregressive (AR) or AR-dominated designs. AR models impose strict causal ordering that is fundamentally mismatched with non-sequential modalities like images. Current alternatives have limitations:

Existing Omni-LLMs struggle with complex real-world emotional scenarios:

memoryllm 的记忆池固定大小（1B 参数），有效知识保持范围仅约 20k token。

部署后的 LLM 参数静态不变，无法高效注入新知识。现有方案（RAG、model editing、长上下文）各有局限：

LLM-based ASR 虽然性能强，但面临三大实际挑战：

统一处理 text + vision + audio + speech 输入/输出的大语言模型范式。代表模型: GPT-4o, qwen3-omni, emova 等。

Building an intelligent, fully interactive agent requires generating coherent multimodal outputs combining high-level reasoning with expressive body motion, speech, and video. Existing systems have critical limitations:

When processing multimodal inputs, models often implicitly assign unequal weights to different modalities. In traditional Vision-Language Models (VLMs), this is broadly perceived as "text-dominance bias." However, for native omni-modal LLMs (OLLMs) that integrate text, image, aud...

推理不一定要在 token 空间进行——可以在连续 embedding 空间做"潜在推理"。

传统多模态推理是纯文本的（看图→想→输出文字）。Omni-R1 提出了 生成式多模态推理：在推理过程中生成中间图像（放大区域、标注框、辅助线、视觉预测），让推理本身变成多模态的。

Modern MLLMs generate fluent responses from interleaved text, image, audio, and video inputs, but identifying which input sources support each generated statement remains an open challenge. Existing attribution methods have three critical limitations:

表面上"统一"的多模态模型（UMMs），在信息论视角下存在 双重分歧 (dual divergence)：

多模态推理的两阶段自我改进：

multimodal-latent-reasoning 的后续工作：在潜在推理中引入视觉增强的深度缩放。

对于 Agentic/Reasoning/Coding LLM，更少的数据/参数/训练步骤可能更好——关键在于数据质量和训练策略。

SWE Agent 能达到多高分辨率取决于信息获取。现有评测无法区分是能力不足还是信息不足。

传统软件工程范式如何适应 AI Agent 时代？Agent 系统的开发/测试/部署流程与传统软件有何根本不同？

SWE Agent 在长上下文中进行复杂推理时，如何动态管理推理上下文？

SWE Agent 训练从 execution-free（纯 SFT）走向 execution-based（带环境反馈的 RL）。

将 Process Reward Model (PRM) 应用于代码 Agent：不仅评估最终结果，还评估每一步代码生成的质量。

The Atari 2600 game console serves as a foundational benchmark environment in reinforcement learning research. Its suite of games (via the Arcade Learning Environment) provides diverse challenges for evaluating RL algorithms, including policy-distillation and deep-q-network metho...

Omni-modal LLMs excel at temporally aligned tasks (captioning, QA) but struggle to perceive cross-modal conflicts — a fundamental human capability critical for trustworthy AI. Existing benchmarks have two major limitations:

Cursor 内部 agentic software engineering 评测套件，从真实 Cursor 编码会话中提取任务。

缺乏系统性评估 LLM 在长期多会话对话中记忆能力的 benchmark。

现有长期记忆 benchmark 两大不足：

LLM 预训练本质是 有损压缩 (lossy compression)，训练数据的相关信息被保留，无关信息被丢弃。

预训练 LLM 时，是否应该考虑下游 RAG 使用场景？传统预训练将 memorization 和 retrieval 视为独立问题。

在 Aurora 超级计算机上大规模 MoE LLM 预训练的工程实践和 scaling 经验。

Formal Statement. Let $\pi_\theta$ be a policy model being pretrained. Standard pretraining optimizes next-token prediction: $\max_\theta \sum_{t} \log \pi_\theta(x_t | x_{<t})$. This ignores desirable properties (safety, factuality, quality) that are only addressed during post-t...

现有 synthetic data 重写只操作单文档，无法建立跨文档关联。

从被动响应 (reactive) → 主动介入 (proactive)：从用户行为推断潜在需求，在实时约束下主动行动。

LLM Agent 的推理轨迹被当作可靠内部信念，但连贯推理仍可能违反逻辑/证据约束，

VerificAgent (arXiv:2506.02539) 是 Microsoft 提出的计算机使用 Agent 记忆验证框架，通过专家种子知识、迭代记忆增长和人工事实核查三阶段流程，系统性地构建经过验证的 Agent 记忆库。在 OSWorld Office 基准上，任务完成率从 25% 提升到约 50%。

*对开放平台 AI Agent 进行系统性状态投毒攻击与防御评估 (2026-04)*

Omni-modal Large Language Models (Omni-MLLMs) promise unified integration of diverse sensory streams, yet a critical performance paradox exists: unimodal baselines frequently outperform joint multimodal inference. This perceptual fragility stems from the static fusion topologies ...

Mixture of Experts (MoE) is a neural network architecture that routes tokens to different "expert" sub-networks, enabling larger model capacity without proportionally increasing compute per forward pass. MoE architectures are used in models like qwen3 and composer2, where they pr...

Shortcut-connected MoE 是 longcat-flash 的另一核心架构创新，通过重排执行流水线来扩大计算-通信重叠窗口，显著提升训练和推理效率。

零计算专家是 longcat-flash 架构的核心创新之一，一种在 Mixture-of-Experts (MoE) 模型中实现动态计算预算分配的机制。

Audio-Agent 是指能够直接以音频为输入、通过 ReAct（Reasoning + Acting）循环执行 Function Call 和 Tool 调用的智能体系统。区别于"ASR + Text Agent"的级联方案，Audio-Agent 从音频到动作端到端完成，保留语气、情感、停顿等非语言信息。

MiniCPM-o 4.5 是一个 9B 参数的开源多模态大语言模型（MLLM），实现了实时全双工 omni-modal 交互——能同时看、听、说，并表现出主动行为（proactive behavior）。其核心技术是 Omni-Flow，一个将多模态输入输出沿共享时间轴对齐的统一流式框架。

基于 Whisper 的领域适配音频编码器，用于高效 Audio-LLM 训练。

语音交互系统中，声学前端（降噪、回声消除、波束成形等）和后端 ASR 之间应该如何分工？

Game-theoretic measurement of AI self-awareness uses strategic games to test whether LLMs can differentiate their reasoning based on opponent identity—including when the opponent is described as being "like you." This approach provides a behavioral, quantitative framework for mea...

Introspection in the context of large language models refers to the capability of an LLM to examine and report on its own internal decision-making processes and learned behaviors. This concept is central to behavioral-self-awareness research, where models demonstrate the ability ...

Mental self-modeling refers to the capacity of large language models (LLMs) to represent and reason about their own internal decision-making processes, mental states, and behavioral dispositions. It is a cognitive capacity related to—but distinct from—behavioral-self-awareness, t...

Situational awareness in AI systems refers to a model's ability to understand its own context, state, capabilities, and the broader environment in which it operates. This concept is related to behavioral-self-awareness but is broader, encompassing not just knowledge of learned be...

Large Audio-Language Models (LALMs) have set new benchmarks in speech processing, but their deployment is hindered by the KV cache memory footprint during long-context inference. The KV cache grows linearly with audio duration, severely limiting on-device and streaming deployment...

Large Audio-Language Models 在长音频推理时 KV cache 占用巨大。

Formal Statement. During autoregressive inference, a Transformer with $L$ layers stores a KV cache of size:

Thinking budget refers to the computational allocation for reasoning steps in models that support extended thinking or chain-of-thought processing. Models like qwen3 allow dynamic adjustment of thinking budget, enabling users to trade off between response latency and reasoning de...

可扩展的合成数据生成方法，用于扩散模型训练。通过混合多种数据源和技术生成高质量训练数据。

用 influence function 引导合成数据的 rubrics（规则/模板）优化：

评估 LLM 作为多语言合成数据生成器的能力：哪些语言、哪些模型最适合做"多语言教师"。

用强化学习引导合成数据生成过程，在保护隐私的同时最大化下游任务效用。

EAGLE 是一种 外挂模型 (External Draft Model) 的投机解码技术。它训练一个轻量级的 Draft Model 来“模仿”Target Model 的生成风格，从而并行生成多个候选 token，再由 Target Model 验证。

MTP (Multi-Token Prediction) 是一种 原生内置 (Native) 的投机解码 (Speculative Decoding) 技术。与外挂的 Draft Model 不同，MTP 将预测头直接嵌入在 Transformer 的中间层中，在预训练阶段就学习预测多个未来的 token。

Speculative Decoding 是一种 LLM 推理加速技术，通过轻量级 draft model 提前生成候选 token，由 target model 并行验证，实现无损加速。

Series: Self-Taught RLVR 系列第三篇（前作：RLSD / NPO）

GPT-Driver 框架将驾驶场景表示为语言 prompt，用 CoT 生成 waypoint 轨迹。

标准 OPD 对所有 rollout 统一应用 KL 监督，忽略了轨迹间信号质量的根本差异。SCOPE 按 轨迹正确性 路由到两条互补监督路径。

Chain-of-Thought (CoT) reasoning is a prompting technique where language models generate intermediate reasoning steps before producing a final answer. CoT prompting dramatically improves performance on complex reasoning tasks and underpins modern reasoning models like those in qw...

Out-of-context reasoning (OOCR) refers to the ability of large language models to reason about information that is not explicitly present in their current context, drawing on knowledge or behavioral patterns acquired during training. This capability is central to behavioral-self-...

STaR (Self-Taught Reasoner) is a method where language models generate their own reasoning traces (chain-of-thought) to solve problems, then fine-tune on their own successful reasoning. This creates a self-improvement loop that enhances reasoning capabilities without external tea...

AI Agent 自主完成软件工程任务的模式。区别于传统的 code completion 或 chat 式编码助手，agentic coding 要求模型能够：

Code/Data: HuggingFace | GitHub | EasyDistill

Model: https://huggingface.co/PolarSeeker/OpenSeeker-v2-30B-SFT

|# Exposure Bias in LLMs

Multi-Task Learning (MTL) is a learning paradigm where a single model is trained to perform multiple tasks simultaneously, leveraging shared representations to improve generalization. In the context of policy-distillation, multi-task learning is relevant because distillation ofte...

Video-to-Audio generation (V2A) aims to synthesize semantically consistent, temporally aligned audio conditioned on video. Despite rapid progress, prior V2A models have three critical limitations:

Behavioral self-awareness is the ability of a large language model (LLM) to explicitly articulate its own learned behaviors without requiring in-context examples or special training for self-description. This is a specific form of out-of-context reasoning (OOCR).

Emergent misalignment is a phenomenon in which fine-tuning a large language model on a seemingly benign, narrow task inadvertently induces broadly misaligned and harmful behavior—far beyond the specific training domain. The term was coined by Betley et al. (2025), who demonstrate...

Data Laundering: 通过合成/改写掩盖原始数据来源，将有问题的数据"洗白"后混入训练集。

GPU 永久性故障会导致 LLM 训练中的静默数据损坏 (Silent Data Corruption, SDC)——数据损坏但训练不报错。

GRPO (Group Relative Policy Optimization) is a reinforcement learning training method that optimizes policy by comparing relative rewards within groups of samples, rather than relying on absolute reward values. GRPO is widely adopted in modern LLM post-training and is referenced ...

GRPO-based RL training methodology refers to the application of Group Relative Policy Optimization in the reinforcement learning phase of LLM post-training. This approach is used in agentic coding systems like agentic-coding and models such as composer2, where GRPO provides a sta...

Contrastive learning is a core component of modern dense retrieval systems, but its effectiveness heavily relies on the quality of negative examples used during training. While hard negative mining (HNM) consistently improves offline metrics, a fundamental question remains: do of...

Formal Statement. Multivector retrieval models (e.g., ColBERT) represent documents as sets of token-level embeddings $\{x_{d,1}, \dots, x_{d,n_d}\} \subset \mathbb{R}^d$. Relevance is computed via late interaction (MaxSim):

Free Process Rewards refers to the idea of obtaining process-level reinforcement learning reward signals without requiring manual annotation. This concept is explored in generalized-on-policy-distillation as a way to provide richer training signals for on-policy distillation. By ...

Reinforcement Learning from Human Feedback (RLHF) is a post-training technique that aligns language models with human preferences by training a reward model on human preference data and then optimizing the language model policy against this reward using reinforcement learning. RL...

Backdoor awareness refers to the ability of large language models to detect or articulate the presence of backdoor triggers planted in their training data. This concept is related to behavioral-self-awareness, as it requires the model to reason about its own learned behaviors — s...

*Benchmark for long-horizon memory safety (2026)*

数据集构造：

多轮 Agent 场景下的 self-distillation：按 skill 类型条件化蒸馏，让 agent 在不同能力维度上独立学习。

Formal Statement. Given a text $t_i$ with token embeddings $\mathbf{X}_i = [\mathbf{x}_{i,1}, \ldots, \mathbf{x}_{i,n_i}] \in \mathbb{R}^{d \times n_i}$, mean pooling produces:

首个端到端统一 音频理解 + 生成 + 编辑 的框架，覆盖通用声音、音乐、语音三大领域。

将 RL 后训练范式扩展到全模态模型，用 时间重排 (temporal reordering) 作为自监督代理任务。

LLM 后训练的本质是 结构化的行为干预 (structured intervention on model behavior)，而非孤立的方法选择。论文提出按 trajectory provenance（轨迹来源）组织整个后训练领域。

现有 Generative Reward Models (GRMs) 只关注 outcome-level supervision，忽略了分析过程质量。

基于认识论 (epistemology) 原则增强 LLM 的自学习对话：让模型区分"知道"、"不知道"、"不确定"三种状态。

*将长上下文推理分解为原子技能，用强化学习逐一提升 (2026-04)*

*半形式化推理模板让 LLM Agent 在代码分析任务中持续超越标准推理 (2026-03)*

不完美验证器也足够好 —— 系统性研究 RLVR 中奖励噪声的影响，发现 15% 噪声率下训练效果无损，精度比召回率更重要。

复杂损失函数是否必要？ —— 系统性拆解 GRPO 组件，提出简化变体 RGRA 在 17/27 基准上超越 GRPO。

*语义密度原则：压缩不总是好事，零信息 token 才是优化目标 (2026-04)*

Code-A1 通过将 Code LLM 与 Test LLM 分离为独立模型进行对抗协同训练，解决了自博弈中自共谋 (self-collusion) 的核心问题，生成的测试质量可替代人工标注。

Agent 与训练数据共同进化 —— 闭环反馈驱动的 Agent RL 训练框架，通过遗忘/边界/稀少信号自动发现弱点并合成新任务。

概要: 本文通过构建 CrossMath 基准——一个支持 text-only / image-only / image+text 三种严格等价输入格式的多模态推理数据集——系统性地揭示了 VLM 在视觉推理上的根本不足。实验发现 SOTA VLM 在文本输入下表现优异，但加入视觉数据后性能反而下降。通过 SFT+GRPO 后训练可有效缓解此差距。

GASP 提出通过真实困难样本作为"标杆"引导教师模型生成更有意义的训练题，解决了非对称自博弈中探索效率低下的核心瓶颈。

逐步注意力蒸馏 + 混合层对齐 —— 首次将教师模型的逐步注意力模式迁移至学生模型，通过 MoL 动态层映射突破蒸馏中的架构差异限制。

首个将 RAG 集成到全双工语音语言模型的系统。核心思想：利用全双工对话中"回复开头"到"关键信息出现"之间的天然时间差（keyword delay），在此期间异步触发外部知识检索，检索结果在关键信息到达前注入模型，从而在不破坏实时交互的前提下大幅提升事实性。基于 moshi 7B 模型，检索后端可插拔（Gemma 3 27B / GPT-4.1 / Tavily 搜索），无需重新训练即可切换。

Outcome Reward 不保证推理链可验证或因果重要 —— 对 RLVR 后训练根本性质的系统性质疑与改进方案。

PARM 提出面向多阶段 LLM 流水线的自适应奖励模型训练方法，利用流水线最终执行反馈自动构建偏好数据，无需人工标注即可训练出有效的阶段级奖励模型。

概要: PieceHint 提出了一种基于价值驱动的问题增强框架，通过识别推理中的关键瓶颈步骤、按问题难度分配提示、并渐进式撤回支架，使小模型（1.5B）在六项数学推理基准上达到与 32B 基线可比的平均性能，同时保持 pass@k 探索多样性。

RLCM 通过在推理轨迹的中间状态上施加边际 (margin) 化校准奖励，使模型在保持推理准确率的同时显著降低过度自信，整体 PCE 从 0.065 降至 0.036。

概要: ReasonXL 提出了一套系统化方案，通过大规模多语言平行推理语料库和 SFT+RLVR 两阶段后训练流水线，将 LLM 的推理语言从英语转移到目标语言（德/法/意/西），同时保持甚至提升任务性能。该工作还通过权重和激活层面分析揭示了模型内部语言编码的机理。

DataPRM 提出了一种环境感知的过程奖励模型，通过 ReAct 范式与代码执行环境交互来验证数据分析每一步的正确性，仅用 4B 参数即超越 72B 级通用 PRM。

概要: 本文系统性地挑战了当前 VLM（视觉-语言模型）忠实整合视觉与文本信息的假设。通过提出 Modality Translation Protocol 和三个量化指标（ToS / CoS / FoS），揭示了 SOTA VLM 在视觉推理中的功能性盲区。文章进一步提出 Divergence Law of Multimodal Scaling 假说和 Semantic Sufficiency Criterion（SSC），为可信多模态推理奠定理论基础。

*从维基百科表格出发的全开源长上下文推理数据管线 (2026-04)*