---
title: OPSDC On-Policy Self-Distillation Reasoning Compression
arxiv_id: 2603.05433
source: https://arxiv.org/html/2603.05433
---

 

 CRISP: Compressed Reasoning via Iterative Self-Policy Distillation 

Report GitHub Issue

 × 

 Title: 

Content selection saved. Describe the issue below:

 Description: 

 Submit without GitHub 
 Submit in GitHub 

 Back to arXiv 

 Why HTML? 

 Report Issue 

 Back to Abstract 

 Download PDF 

 Abstract 

 1 Introduction 

 Summary of results. 

 2 Related Work 

 Reasoning compression via reinforcement learning. 

 Reasoning compression via supervised fine-tuning. 

 Training-free compression. 

 On-policy self-distillation. 

 3 Method 

 3.1 Problem Formulation 

 3.2 Training Objective 

 Why reverse KL? 

 3.3 Teacher Parameterization 

 Difficulty-adaptive compression. 

 3.4 Training Algorithm 

 Computational cost and simplicity. 

 4 An Interpretive Framework 

 Implicit reward. 

 Accuracy preservation (Proposition  1 ). 

 Difficulty-adaptive compression (Proposition  2 ). 

 Bounded forgetting (Proposition  3 ). 

 Compression reduces compounding error (Proposition  4 ). 

 5 Experiments 

 5.1 Experimental Setting 

 Models and data. 

 Benchmarks. 

 5.2 Self-Distillation Simultaneously Compresses and Improves Reasoning 

 5.3 Compression Naturally Adapts to Problem Difficulty 

 Larger models gain more from distillation. 

 5.4 Self-Distillation Does Not Collapse Model Entropy 

 5.5 Why Does Compression Improve Accuracy? 

 5.6 CRISP Generalizes Across Model Families 

 Why is compression smaller than on Qwen models? 

 6 Limitations and Future Work 

 Instruction-following as an enabler. 

 Teacher quality characterization. 

 7 Conclusion 

 References 

 A Theoretical Analysis 

 A.1 Preliminary: Sequence-Level Divergence 

 A.2 Preliminary: Implicit Reward Interpretation 

 A.3 Accuracy Preservation under Compression 

 A.4 Difficulty-Adaptive Compression 

 A.5 Bounded Forgetting from the Base Model 

 A.6 Compression Reduces Compounding Error 

 Quantitative interpretation. 

 B Prompt Templates 

 C Ablation Study 

 C.1 Do Quantitative Reduction Targets Outperform Qualitative Instructions? 

 C.2 How Sensitive Is Compression to the Teacher Update Interval? 

 D Survey of Reasoning Compression Methods 

 E Extended Results Across Token Budgets 

 E.1 Results Under 8K Token Budget 

 Truncation explains the amplified accuracy gains. 

 E.2 Extended Training Under 30K Token Budget 

 Practical recommendation: step 100 is the sweet spot. 

 Difficulty-dependent over-compression. 

 F Training and Implementation Details 

 Technical setup. 

 Training data. 

 Training procedure. 

 Hyperparameters. 

 Evaluation. 

 G Alternative Teacher Parameterizations 

 Frozen teacher ( M = ∞ M=\infty ). 

 Periodic teacher (our default, M = 50 M{=}50 ). 

 EMA teacher. 

 Stop-gradient concurrent teacher ( M = 1 M{=}1 ). 

 H Token Reduction and Accuracy over Training 

 H.1 Entropy Stability Throughout Training 

 I Effect of KL Divergence Direction in CRISP 

 I.1 Background and Motivation 

 I.2 Experimental Setup 

 I.3 Results 

 I.4 Mechanism and Conclusion 

 J Deep Planning Agentic Task 

 Travel Planning. 

 Shopping Planning. 

 Setup. 

 Results. 

 Shopping quality degrades after step 70. 

 Absolute scores reflect benchmark difficulty, not model weakness. 

 Limitations. 

 K Qualitative Examples 

 License: CC BY 4.0

arXiv:2603.05433v6 [cs.LG] 11 Apr 2026

CRISP: Compressed Reasoning via Iterative Self-Policy Distillation

Hejian Sang 

 hejian@alumni.iastate.edu 
&Yuanda Xu 1 1 footnotemark: 1 

 yuanda@math.princeton.edu 
&Zhengze Zhou 1 1 footnotemark: 1 

 zz433@cornell.edu 
&Ran He 1 1 footnotemark: 1 

 rh2528@columbia.edu 
&Zhipeng Wang 

 zhipeng.wang@alumni.rice.edu 
&Jiachen Sun 

 jiachens@umich.edu 
 Equal contribution.Correspondence to hejian@alumni.iastate.edu 

Abstract

Reasoning models think out loud, but much of what they say is noise. We introduce CRISP ( C ompressed R easoning via I terative S elf- P olicy Distillation), a method that teaches models to reason more concisely by distilling their own concise behavior back into themselves. The entire approach reduces to one idea: condition the same model on a “be concise” instruction to obtain teacher logits, and minimize per-token reverse KL on the student’s own rollouts. No ground-truth answers, no token budgets, no difficulty estimators. Just self-distillation. Yet this simplicity belies surprising sophistication: CRISP automatically compresses easy problems aggressively while preserving the deliberation needed for hard ones. On Qwen3-8B and Qwen3-14B, we achieve 57–59% token reduction on MATH-500 while improving accuracy by 9–16 points absolute. On AIME 2024, the 14B model gains 10 points with 41% compression. Ablations show that qualitative conciseness instructions outperform explicit token targets, and periodic teacher refreshes yield a broad stable regime. The method generalizes across model families—DeepSeek-R1-Distill-Llama-8B improves accuracy by up to 5 points with 17–32% compression—and transfers beyond math to multi-step agentic planning (DeepPlanning), reducing token usage by 42–51% while preserving planning quality.

 Figure 1: The paradox of reasoning compression: less thinking, better answers. Results for Qwen3-14B across three benchmarks of increasing difficulty (30K response token budget). CRISP compresses reasoning traces by 35–57% while largely preserving or improving accuracy, most dramatically on MATH-500, where accuracy jumps from 70.0% to 86.1%. 

 1 Introduction

Modern reasoning models have learned to think before they speak, and they have a lot to say. Systems like OpenAI o1  (Jaech et al. , 2024 ) , Gemini 2.5  (Comanici et al. , 2025 ) , DeepSeek-R1  (Guo et al. , 2025 ) , and Qwen3  (Yang et al. , 2025 ) produce thousands of tokens of internal deliberation before arriving at an answer, exploring blind alleys, second-guessing themselves, and verifying conclusions. This verbosity pays off on hard problems. But it comes at a cost: these models cannot stop talking , even when the answer is obvious. Ask them what 2 + 2 2+2 is, and they may spend 500 tokens considering whether you meant binary arithmetic  (Snell et al. , 2024 ; Muennighoff et al. , 2025 ) . That being said, the computational overhead stem from the "overthinking" does not help on the quality of answers for many problems.

The community has noticed the limitations. A flurry of reasoning compression methods has emerged (Appendix  D surveys some recent approaches), each attacking the problem from a different angle. But every existing paradigm demands a sacrifice: RL methods need ground-truth answers and risk collapsing the model’s ability to explore  (Aggarwal and Welleck, 2025 ; Wan et al. , 2026 ; Liu et al. , 2025 ) ; SFT methods train on someone else’s reasoning and forget their own  (Huang et al. , 2025 ; Shenfeld et al. , 2026 ) ; most treat all problems alike, compressing a trivial sum as aggressively as a competition integral; and prompting tricks vanish the moment you remove the prompt.

We propose CRISP ( C ompressed R easoning via I terative S elf- P olicy Distillation), a method that sidesteps all of these trade-offs with a single, almost trivial idea: ask the model to be concise, then teach it to do so without being asked . The model already knows how to compress; it just needs permission. We give it that permission via a conciseness instruction, then distill this behavior back into the base model. No rewards, no budgets, no oracles. Given a reasoning model π θ \pi_{\theta} , we define:

 • 

 Teacher : π θ ( ⋅ ∣ x , c ) \pi_{\theta}(\cdot\mid x,c) , the same model conditioned on a conciseness instruction c c (for example: “Solve concisely, avoid unnecessary steps”).

 • 

 Student : π θ ( ⋅ ∣ x ) \pi_{\theta}(\cdot\mid x) , the same model without the compression instruction.

Training generates student rollouts and minimizes the per-token reverse KL divergence between student and teacher distributions. This on-policy self-distillation approach requires no ground-truth answers, no reward engineering, and no difficulty estimation. The compression signal emerges naturally from the KL objective, adapting automatically to problem difficulty.

 Table 1: Comparison of reasoning compression methods. CRISP uniquely combines on-policy training, no dependence on ground-truth (GT) answers, difficulty-adaptive compression, and entropy preservation. 

 Method 

 On- 

 policy 

 No GT 

 needed 

 Difficulty- 

 adaptive 

 Entropy- 

 preserving 

 RL + length penalty (Aggarwal and Welleck, 2025 ; Wan et al. , 2026 ) 

 ✓ 
 ✗ 
 ✗ 
 ✗ 

 SFT on compressed CoT (Huang et al. , 2025 ) 

 ✗ 
 ✗ 
 ✗ 
 ✓ 

 OPCD (Ye et al. , 2026 ) 

 ✓ 
 ✗ 
 ✗ 
 ✓ 

 DLER (Liu et al. , 2025 ) 

 ✓ 
 ✗ 
 ✗ 
 ✗ 

 Prompting / pruning (Xu et al. , 2025 ) 

 — 
 ✓ 
 ✗ 
 ✓ 

 CRISP (ours) 
 ✓ 
 ✓ 
 ✓ 
 ✓ 

Table  1 contrasts CRISP with representative methods from each paradigm. CRISP is the only approach that satisfies all four desiderata.

Summary of results.

On Qwen3-8B and Qwen3-14B, CRISP achieves 57–59% token reduction on MATH-500 while improving accuracy by 9–16 percentage points (to ∼ {\sim} 86%). On AIME 2024, the 14B model gains 10 points with 41% compression. Compression naturally adapts to difficulty ( ∼ 1.6 × {\sim}1.6\times more compression on easy vs. hard problems), entropy remains stable throughout training, and general capabilities (MMLU) are fully preserved. Our ablations also show that the default recipe is principled rather than accidental: qualitative “be concise” instructions outperform explicit percentage targets, periodic teacher refreshes admit a broad stable regime at M ∈ { 40 , 50 , 60 } M\in\{40,50,60\} , and reverse KL is markedly more stable than forward KL for iterative on-policy self-distillation. Beyond math benchmarks, the method generalizes in two important dimensions. Cross-model: applying the same recipe to DeepSeek-R1-Distill-Llama-8B (a different model family) yields accuracy gains of up to 5 points with 17–32% token reduction, confirming that the approach is not specific to Qwen. Cross-task: the compressed Qwen3-14B model transfers to multi-step agentic planning on DeepPlanning, reducing response length by 42–51% while preserving planning quality.

 2 Related Work

Reasoning compression via reinforcement learning.

The most direct approach: penalize length in the reward function. L1  (Aggarwal and Welleck, 2025 ) caps token count during GRPO training. DiPO  (Wan et al. , 2026 ) and DIET  (Chen et al. , 2025b ) estimate difficulty from rollout pass rates and set per-problem length targets. Leash  (Li et al. , 2025b ) shapes rewards with sigmoid functions; DLER  (Liu et al. , 2025 ) adds curriculum learning. ThinkPrune  (Hou et al. , 2025 ) continuously trains long-thinking LLMs using reinforcement learning (RL) with an additional token-budget constraint while preserving answer correctness. The catch: all of these require ground-truth answers. No correct answer, no reward and no way to know if compression went too far.  Xu et al. ( 2026a ) further notes that reinforcement learning can induce overconfidence errors, thereby narrowing the model’s reasoning boundary and reducing generation diversity.

Reasoning compression via supervised fine-tuning.

Another route: curate short reasoning traces, then train on them. SEER  (Huang et al. , 2025 ) samples many solutions and keeps the shortest correct ones. TokenSkip  (Xia et al. , 2025 ) learns which tokens to skip. DAP/LiteCoT  (Wu et al. , 2025 ) distills from stronger models; S3-CoT  (Du et al. , 2026 ) steers activations toward brevity. The problem is distribution shift: the student trains on someone else’s reasoning and forgets its own  (Shenfeld et al. , 2026 ) .

Training-free compression.

The lightweight option: change the prompt or the decoder, not the weights. Chain of Draft  (Xu et al. , 2025 ) asks for minimal drafts instead of full reasoning. TrimR  (Lin et al. , 2025 ) prunes after the fact. NoWait  (Wang et al. , 2025a ) and FlowSteer  (Li et al. , 2026 ) steer decoding toward conciseness. These methods are easy to deploy but achieve limited compression, and the effect vanishes when you change the prompt.

On-policy self-distillation.

The closest relatives of our work use the model as its own teacher. OPSD  (Zhao et al. , 2026 ) gives the teacher the ground-truth answer, achieving 4–8 × \times efficiency over GRPO. SDPO  (Hübotter et al. , 2026 ) conditions on rich feedback for dense credit assignment. SDFT  (Shenfeld et al. , 2026 ) shows that on-policy distillation dramatically reduces forgetting compared to standard SFT, interpreting it as inverse RL. PACED  (Xu et al. , 2026b ) studies competence-aware self-distillation via pass-rate weighting. OPCD  (Ye et al. , 2026 ) distills system-prompt behaviors into weights. We contribute a new application: using a conciseness instruction as the privileged context, achieving compression without any ground-truth supervision.

 3 Method

 3.1 Problem Formulation

Consider a reasoning model π θ \pi_{\theta} that, given input x x , generates a reasoning trace r r followed by an answer a a , producing output y = ( r , a ) y=(r,a) . The reasoning trace typically appears within <think> … \ldots </think> delimiters. We aim to learn parameters θ ∗ \theta^{*} such that the model produces shorter reasoning traces while maintaining accuracy.

Let c c denote a conciseness instruction. The student receives the original DAPO-17K math prompt, while the teacher receives the same prompt prefixed with a conciseness instruction. We denote them by π θ ( ⋅ ∣ x ) \pi_{\theta}(\cdot\mid x) and π θ ( ⋅ ∣ x , c ) \pi_{\theta}(\cdot\mid x,c) ; full templates are provided in Appendix  B .

 3.2 Training Objective

CRISP minimizes the per-token reverse KL divergence between the student and a stop-gradient teacher on student-generated rollouts:

 ℒ ( θ ) = 𝔼 x ∼ 𝒟 , y ∼ π θ ( ⋅ ∣ x ) [ ∑ t = 1 | y | D KL ( π θ ( ⋅ ∣ x , y < t ) ∥ π θ ¯ ( ⋅ ∣ x , c , y < t ) ) ] , \mathcal{L}(\theta)=\mathbb{E}_{x\sim\mathcal{D},\;y\sim\pi_{\theta}(\cdot\mid x)}\left[\sum_{t=1}^{|y|}D_{\mathrm{KL}}\Big(\pi_{\theta}(\cdot\mid x,y_{<t})\;\Big\|\;\pi_{\bar{\theta}}(\cdot\mid x,c,y_{<t})\Big)\right], 

 (1) 

where θ ¯ \bar{\theta} denotes the teacher weights, which are periodically synchronized with the student (Section  3.3 ), and no gradients flow through the teacher’s forward pass. The expectation over y ∼ π θ ( ⋅ ∣ x ) y\sim\pi_{\theta}(\cdot\mid x) makes training on-policy : the student is optimized on its own generation distribution, which prevents the distribution shift inherent in off-policy SFT.

Why reverse KL?

The divergence direction matters because teacher refreshes make training iterative rather than one-shot. Reverse KL ( D KL ​ ( π θ ∥ π θ ¯ ) D_{\mathrm{KL}}(\pi_{\theta}\|\pi_{\bar{\theta}}) ) weights updates by the student’s own distribution, so the model only changes in regions it already visits, which stabilizes periodic teacher refreshes. Forward KL instead weights updates by the teacher distribution and empirically overreacts to refreshes, causing aggressive over-compression and accuracy collapse on hard benchmarks (Appendix  I ).

 3.3 Teacher Parameterization

A natural baseline is a fully frozen teacher ( θ ¯ = θ 0 \bar{\theta}=\theta_{0} ) as in Zhao et al. ( 2026 ) . While simple and stable, the frozen teacher becomes an increasingly weak compression target as the student improves: once the student has internalized the initial conciseness signal, no further compression is possible because the reference distribution no longer leads the student.

To address this, we adopt a periodic teacher update strategy. The teacher weights are synchronized with the current student weights every M M training steps:

 θ ¯ ← θ every  ​ M ​  steps . \bar{\theta}\leftarrow\theta\quad\text{every }M\text{ steps}. 

 (2) 

Each refresh creates a new, stronger compression target: the updated teacher, when conditioned on the conciseness instruction c c , produces traces that are more concise than the previous teacher’s (since the student, now serving as the new teacher, has already learned to compress). This progressive compression effect pushes the student to continuously shorten its reasoning over the course of training, beyond what a single frozen reference can achieve.

Difficulty-adaptive compression.

Compression adapts naturally to problem difficulty: for easy problems, the concise teacher produces much shorter traces, creating strong KL signal; for hard problems, even the teacher needs extensive reasoning, yielding weak signal. We formalize this in Proposition  2 and verify it empirically in Section  5.3 .

 3.4 Training Algorithm

The complete CRISP training procedure is given in Algorithm  1 .

 Input: Model π θ \pi_{\theta} , dataset 𝒟 = { x i } \mathcal{D}=\{x_{i}\} , conciseness instruction c c , learning rate η \eta , teacher update interval M M 

 Output: Compressed reasoning model π θ ∗ \pi_{\theta^{*}} 

Initialize teacher: θ ¯ ← θ 0 \bar{\theta}\leftarrow\theta_{0} ;

 for each training step k = 1 , 2 , … k=1,2,\ldots do 

 if k mod M = 0 k\bmod M=0 then 

Update teacher: θ ¯ ← θ \bar{\theta}\leftarrow\theta ;

           // periodic refresh 

      end if

     Sample batch { x 1 , … , x B } ∼ 𝒟 \{x_{1},\ldots,x_{B}\}\sim\mathcal{D} ;

 for each x i x_{i} in batch do 

Generate student rollout: y i ∼ π θ ( ⋅ ∣ x i ) y_{i}\sim\pi_{\theta}(\cdot\mid x_{i}) ;

 for each token position t = 1 , … , | y i | t=1,\ldots,|y_{i}| do 

Compute student logits: q t ← π θ ( ⋅ ∣ x i , y i , < t ) q_{t}\leftarrow\pi_{\theta}(\cdot\mid x_{i},y_{i,<t}) ;

Compute teacher logits: p t ← π θ ¯ ( ⋅ ∣ x i , c , y i , < t ) p_{t}\leftarrow\pi_{\bar{\theta}}(\cdot\mid x_{i},c,y_{i,<t}) ;

                // no grad 

Compute D KL ​ ( q t ∥ p t ) D_{\mathrm{KL}}(q_{t}\|p_{t}) ;

           end for

           ℒ i ← ∑ t D KL ​ ( q t ∥ p t ) \mathcal{L}_{i}\leftarrow\sum_{t}D_{\mathrm{KL}}(q_{t}\|p_{t}) ;

      end for

     Update student: θ ← θ − η ​ ∇ θ 1 B ​ ∑ i ℒ i \theta\leftarrow\theta-\eta\nabla_{\theta}\frac{1}{B}\sum_{i}\mathcal{L}_{i} ;

     ;

      // normalized by | y i | |y_{i}| in practice 

 end for

 return π θ ∗ \pi_{\theta^{*}} ;

 Algorithm 1 CRISP: On-Policy Self-Distillation for Concise Reasoning 

Computational cost and simplicity.

The pipeline requires only standard supervised training infrastructure—no reward models, no value functions, no multi-rollout sampling. Each step requires two forward passes per rollout token (student with gradient, teacher without), and the periodic refresh is a simple weight copy. This yields substantial efficiency gains over RL methods, which require multiple rollouts per prompt, reward model inference, and complex optimization (e.g., PPO clipping, GAE).

 4 An Interpretive Framework

The primary contributions of this paper are methodological and empirical. In this section we complement them with an interpretive framework that makes the mechanisms behind CRISP more transparent. We keep only the main takeaways here and defer all definitions and proofs to Appendix  A . The framework serves four purposes: it identifies the training loss as a sequence-level reverse KL, interprets this KL as an implicit conciseness reward, explains why compression can preserve accuracy and avoid forgetting, and clarifies why shorter traces can sometimes improve accuracy. None of these results require novel proof techniques; their value lies in connecting CRISP’s design choices to well-understood information-theoretic quantities and in guiding practitioner intuition.

By the autoregressive chain rule, the per-token objective in Eq.  1 is exactly the sequence-level divergence D KL ( π θ ( ⋅ ∣ x ) ∥ π θ ¯ ( ⋅ ∣ x , c ) ) D_{\mathrm{KL}}(\pi_{\theta}(\cdot\mid x)\|\pi_{\bar{\theta}}(\cdot\mid x,c)) (Lemma  1 ). This lets us analyze CRISP as matching the student’s full rollout distribution to that of the concise teacher on the student’s own trajectories.

Implicit reward.

The CRISP objective (Eq.  1 ) is equivalent to maximizing the expected implicit reward

 r ​ ( y t , x ) = log ⁡ π θ ¯ ​ ( y t ∣ x , c , y < t ) − log ⁡ π θ ​ ( y t ∣ x , y < t ) . r(y_{t},x)=\log\pi_{\bar{\theta}}(y_{t}\mid x,c,y_{<t})-\log\pi_{\theta}(y_{t}\mid x,y_{<t}). 

 (3) 

This reward is positive on tokens the concise teacher prefers and negative on tokens the student overproduces. In effect, CRISP suppresses unnecessary reasoning without introducing an explicit length penalty or a correctness verifier.

Accuracy preservation (Proposition  1 ).

If training converges to loss ϵ KL \epsilon_{\mathrm{KL}} and the concise teacher preserves accuracy to within ϵ T \epsilon_{T} of the base model, the student satisfies

 Acc ​ ( π θ ∗ ) ≥ Acc ​ ( π θ ¯ ) − ϵ T − ϵ KL / 2 . \mathrm{Acc}(\pi_{\theta^{*}})\geq\mathrm{Acc}(\pi_{\bar{\theta}})-\epsilon_{T}-\sqrt{\epsilon_{\mathrm{KL}}/2}. 

 (4) 

The bound cleanly separates teacher quality from distillation gap. In our setting, the concise teacher is often more accurate than the base model, so compression can improve accuracy rather than merely preserve it.

Difficulty-adaptive compression (Proposition  2 ).

The compression signal is non-increasing in problem difficulty: easy problems receive stronger pressure to shorten reasoning, while hard problems receive weaker pressure because a larger fraction of their tokens are essential. This formalizes the empirical pattern that CRISP compresses MATH-500 much more aggressively than AIME without any explicit difficulty estimator.

Bounded forgetting (Proposition  3 ).

Divergence from the base model is bounded by

 𝔼 x [ d TV ( π θ ∗ ( ⋅ ∣ x ) , π θ 0 ( ⋅ ∣ x ) ) ] ≤ ϵ KL / 2 + 𝔼 x [ γ ( x ) ] , \mathbb{E}_{x}\big[d_{\mathrm{TV}}(\pi_{\theta^{*}}(\cdot\mid x),\,\pi_{\theta_{0}}(\cdot\mid x))\big]\leq\sqrt{\epsilon_{\mathrm{KL}}/2}+\mathbb{E}_{x}[\gamma(x)], 

 (5) 

where γ ​ ( x ) \gamma(x) is the conciseness gap, i.e., the total variation between the base model’s outputs with and without the conciseness instruction. For hard problems, the instruction changes the base model very little, so γ ​ ( x ) \gamma(x) is small and forgetting is correspondingly limited.

Compression reduces compounding error (Proposition  4 ).

Under a model where each token independently introduces a reasoning error with probability p err p_{\mathrm{err}} , compressing from L L to α ​ L \alpha L tokens yields an accuracy ratio ( 1 − p err ) − ( 1 − α ) ​ L (1-p_{\mathrm{err}})^{-(1-\alpha)L} , which grows exponentially in the number of removed tokens. This gives a simple explanation for the empirical “less is more” effect: when long chains contain redundant or fragile steps, removing tokens can reduce the chance that the model talks itself into an error.

 5 Experiments

 5.1 Experimental Setting

Models and data.

We evaluate CRISP on Qwen3-8B and Qwen3-14B  (Yang et al. , 2025 ) , training on ∼ {\sim} 13,600 competition-level math problems from DAPO-Math-17k  (Yu et al. , 2025 ) without ground-truth answers ; only problem statements are used to generate student rollouts. We train for 1 epoch with learning rate 1 × 10 − 6 1\times 10^{-6} , global batch size 32, periodic teacher update (interval M = 50 M{=}50 ; see ablation in Section  C.2 ), and 8 × 8\times H200 GPUs. Although nominally a full epoch, the algorithm converges quickly at around ∼ {\sim} 100 steps. Each prompt generates a single student rollout (temperature 1.0) with a maximum response length of 8,192 tokens. Because CRISP optimizes a per-token KL objective rather than an outcome-based reward, partial rollouts already provide a useful training signal, unlike RL methods that require complete responses  (Chen et al. , 2025a ) . Full training and infrastructure details are in Appendix  F .

Benchmarks.

We evaluate on three mathematical reasoning benchmarks spanning a wide difficulty range: MATH-500  (Hendrycks et al. , 2021 ) (500 problems, base accuracy 70–78%), AIME 2024 (30 problems, 66–73%), and AIME 2025 (30 problems, 63–67%). 1 1 1 All benchmarks are evaluated using the math answer grading utility from veRL  (Sheng et al. , 2025 ) : https://github.com/verl-project/verl/blob/main/verl/utils/reward_score/math_dapo.py . We define a token budget as the maximum response length allowed during inference, a practical lever for controlling serving cost. We report results under two budgets: 8,192 tokens, representative of efficient serving constraints, and 30,000 tokens, which effectively eliminates truncation and enables fairer accuracy comparison.

 5.2 Self-Distillation Simultaneously Compresses and Improves Reasoning

 Table 2: Self-distillation compresses reasoning traces while improving accuracy without forgetting (token budget = 30K). Results on Qwen3-8B and Qwen3-14B with a 30,000-token budget to eliminate truncation effects. Accuracy (Acc, mean over 8 samples per problem, %), average reasoning token length (Len), and token reduction relative to the base model (Red., %). “Concise prompt” uses the conciseness instruction at inference only (no training); CRISP trains with periodic teacher update ( M = 50 M{=}50 ). The rightmost column reports MMLU  (Hendrycks et al. , 2020 ) accuracy to verify that general capabilities are preserved. Results under the efficient-serving budget (8,192 tokens) are in Table  6 (Appendix  E.1 ). 

 MATH-500 
 AIME 2024 
 AIME 2025 
 MMLU 

 Method 
 Acc 
 Len 
 Red. 
 Acc 
 Len 
 Red. 
 Acc 
 Len 
 Red. 
 Acc 

 Qwen3-8B 

 Base Model 
 77.7 
 4,661 
 — 
 72.5 
 14,170 
 — 
 62.5 
 16,682 
 — 
 73.2 

 Concise prompt 
 80.9 
 2,941 
 36.9% 
 67.5 
 11,589 
 18.2% 
 56.3 
 14,347 
 14.0% 
 — 

 CRISP 
 86.6 
 1,921 
 58.8% 
 69.6 
 9,152 
 35.4% 
 57.1 
 10,726 
 35.7% 
 73.3 

 Qwen3-14B 

 Base Model 
 70.0 
 3,872 
 — 
 65.8 
 12,844 
 — 
 67.1 
 15,642 
 — 
 76.9 

 Concise prompt 
 83.8 
 2,426 
 37.3% 
 68.3 
 9,866 
 23.2% 
 58.3 
 12,831 
 18.0% 
 — 

 CRISP 
 86.1 
 1,686 
 56.5% 
 76.3 
 7,577 
 41.0% 
 61.7 
 10,137 
 35.2% 
 76.9 

Table  2 presents our main results under the 30,000-token budget, which eliminates response truncation for a fair accuracy comparison. 2 2 2 MMLU is evaluated using the Language Model Evaluation Harness  (Gao et al. , 2021 ) : https://github.com/EleutherAI/lm-evaluation-harness . 

 Finding 1: Less is more. CRISP simultaneously improves accuracy and reduces reasoning length on MATH-500, the opposite of the trade-off everyone assumes. Both models reach ∼ {\sim} 86% accuracy (from 77.7% for 8B and 70.0% for 14B) while shedding 57–59% of their tokens. On AIME 2024, the 14B model improves substantially ( 65.8 → 76.3 65.8{\to}76.3 , +10.5pp) with 41% compression. On the harder AIME 2025, accuracy decreases modestly ( ∼ {\sim} 5pp) as the model trades some deliberation for efficiency. 

The concise prompt alone (no training) already improves MATH-500 accuracy while reducing tokens by 14–37%, confirming that the base model’s reasoning contains substantial redundancy; CRISP amplifies both effects. Crucially, MMLU accuracy is fully preserved (73.2 → \to 73.3 for 8B, 76.9 → \to 76.9 for 14B), confirming that self-distillation does not degrade general capabilities.

 5.3 Compression Naturally Adapts to Problem Difficulty

 Finding 2: Compression naturally adapts to difficulty. On the easiest benchmark (MATH-500), CRISP compresses by 56.5–58.8% . On the hardest (AIME 2025), compression drops to 35.2–35.7% , a ∼ 1.6 × {\sim}1.6\times ratio that emerges automatically from the KL objective (see § 3.3 ). 

Larger models gain more from distillation.

Qwen3-14B starts from lower base accuracy on MATH-500 (70.0% vs. 77.7% for 8B) yet achieves comparable post-distillation accuracy (86.1% vs. 86.6%). On AIME 2024, the 14B model improves by 10.4 points while the 8B model slightly declines. Larger models follow instructions better, making the concise teacher a stronger signal, and they have more redundancy to shed.

 5.4 Self-Distillation Does Not Collapse Model Entropy

 Finding 3: Self-distillation preserves what RL destroys. A central concern with reasoning compression is entropy collapse : RL methods with length penalties systematically suppress high-entropy “exploratory” tokens (“Wait,” “Alternatively,” “Let me reconsider…”) that are critical for solving hard problems  (Wang et al. , 2025b ) . Figure  8 (Appendix  H.1 ) shows that CRISP avoids this entirely—entropy remains stable throughout training. The model learns to choose conciseness rather than being forced into it. 

This follows from mode-seeking reverse KL (§ 3.2 ): the student is penalized for placing mass where the teacher assigns low probability, but not for maintaining mass where the teacher is also uncertain. RL length penalties, by contrast, reward shorter outputs regardless of token informativeness.

 5.5 Why Does Compression Improve Accuracy?

 Figure 2: Student mean accuracy on training data increases during self-distillation. Qwen3-8B improves from ∼ {\sim} 52% to ∼ {\sim} 66% and Qwen3-14B from ∼ {\sim} 46% to ∼ {\sim} 72%, despite no correctness reward. The concise teacher’s implicit reward reshapes the student’s output distribution, concentrating probability mass on direct, correct reasoning paths. 

 Finding 4: Compression improves accuracy via implicit reward shaping. The reverse KL objective implicitly rewards concise, correct tokens (Eq.  3 ) and concentrates probability mass on the teacher’s preferred mode. Training-time accuracy increases monotonically (Figure  2 )—from ∼ {\sim} 52% to ∼ {\sim} 66% for Qwen3-8B and ∼ {\sim} 46% to ∼ {\sim} 72% for Qwen3-14B—despite receiving no correctness reward. 

The mode-seeking property of reverse KL (§ 3.2 ) drives the student toward the teacher’s preferred mode  (Gu et al. , 2023 ; Li et al. , 2025a ) , concentrating probability mass on direct, correct reasoning paths. Since each additional token is a potential point of failure  (Chen et al. , 2024 ) —an effect we formalize in Proposition  4 —compression simultaneously shortens traces and reduces error accumulation. Figure  12 in Appendix  K provides side-by-side comparisons.

Appendix  C shows that qualitative “be concise” instructions outperform explicit percentage targets (which compress more but sacrifice accuracy), and that the teacher update interval admits a broad stable regime at M ∈ { 40 , 50 , 60 } M\in\{40,50,60\} , while overly frequent updates ( M = 1 M{=}1 ) cause collapse. Appendix  I further shows that replacing reverse KL with forward KL introduces progressively deeper regressions after each teacher refresh.

 5.6 CRISP Generalizes Across Model Families

To evaluate generalization beyond the Qwen model family, we apply CRISP to DeepSeek-R1-Distill-Llama-8B, a reasoning model distilled from DeepSeek-R1  (Guo et al. , 2025 ) into the Llama-3.1-8B architecture. We use the same training recipe as in Section  5.1 .

 Table 3: CRISP on DeepSeek-R1-Distill-Llama-8B compresses reasoning traces while improving accuracy. Accuracy (Acc, mean over 8 samples per problem, %), average reasoning token length (Len), and token reduction relative to the base model (Red., %). CRISP trains with periodic teacher update ( M = 50 M{=}50 ). 

 MATH-500 
 AIME 2024 
 AIME 2025 

 Method 
 Acc 
 Len 
 Red. 
 Acc 
 Len 
 Red. 
 Acc 
 Len 
 Red. 

 Distill-Llama-8B 

 Base Model 
 61.3 
 3,102 
 — 
 37.9 
 11,259 
 — 
 27.5 
 11,214 
 — 

 CRISP 
 62.7 
 2,104 
 32.2% 
 42.9 
 9,311 
 17.3% 
 32.1 
 9,939 
 11.4% 

 Finding 5: CRISP generalizes across model families and task domains. As shown in Table  3 , applying the identical recipe to DeepSeek-R1-Distill-Llama-8B improves accuracy on all three math benchmarks (+1.4 on MATH-500, +5.0 on AIME 2024, +4.6 on AIME 2025) while reducing tokens by 11–32%. On the non-math DeepPlanning benchmark, compressed Qwen3-14B models reduce agentic response tokens by 42–51% while preserving travel-planning and shopping-planning quality (Figure  11 in Appendix  J ). These results confirm that CRISP is not specific to one model family or to mathematical reasoning. 

Why is compression smaller than on Qwen models?

The 32% token reduction on MATH-500 is notably lower than the 57–59% achieved by Qwen3-8B/14B. We attribute this to two factors. First, Distill-Llama-8B already produces substantially shorter base responses (3,102 tokens vs. 4,661 for Qwen3-8B on MATH-500), leaving less redundancy to remove. Second, the Llama architecture appears less responsive to the qualitative conciseness instruction: because DeepSeek-R1-Distill-Llama-8B was trained on DeepSeek-R1 traces rather than instruction-following data, the gap between its base and concise-prompted distributions is smaller, yielding a weaker distillation signal. Despite these differences, the consistent accuracy gains across all three benchmarks confirm that CRISP transfers across model families, with compression magnitude naturally adapting to the amount of available redundancy.

 6 Limitations and Future Work

Instruction-following as an enabler.

CRISP requires reasonably strong instruction-following ability; smaller models may fail to respond reliably to a conciseness instruction, which likely explains why larger models benefit more. Characterizing the minimum capability threshold is an important open question.

Teacher quality characterization.

Our experiments show that the conciseness-conditioned teacher improves accuracy (Table  2 ), and Proposition  1 provides formal bounds. A finer-grained characterization of when and why conciseness instructions improve versus degrade accuracy across model families would further strengthen understanding of self-distillation dynamics.

 7 Conclusion

CRISP shows that much of what reasoning models produce is not deliberation but noise —and noise compounds. Every unnecessary token is a chance to wander off course, to second-guess a correct answer, to introduce an error that propagates forward. By teaching models to skip the noise, we do not sacrifice depth; we recover it.

Two takeaways stand out. First, verbosity is not caution—it can be a source of compounding error. Second, models already possess a latent ability to be concise; on-policy self-distillation can make this behavior the default without sacrificing entropy or general capabilities.

Finally, CRISP’s supervision is purely behavioral: a conciseness instruction and the model’s own rollouts. This suggests a path to compressing reasoning in domains where ground-truth answers or reliable verifiers are unavailable, as long as the model can follow the desired instruction.

References

 P. Aggarwal and S. Welleck (2025) 
 L1: controlling how long a reasoning model thinks with reinforcement learning .

 arXiv preprint arXiv:2503.04697 .

 Cited by: Table 1 ,
 §1 ,
 §2 .

 W. Chen, V. Kothapalli, A. Fatahibaarzi, H. Sang, S. Tang, Q. Song, Z. Wang, and M. Abdul-Mageed (2025a) 
 Distilling the essence: efficient reasoning distillation via sequence truncation .

 arXiv preprint arXiv:2512.21002 .

 Cited by: §5.1 .

 W. Chen, J. Yuan, T. Jin, N. Ding, H. Chen, Z. Liu, and M. Sun (2025b) 
 The overthinker’s diet: cutting token calories with difficulty-aware training .

 arXiv preprint arXiv:2505.19217 .

 Cited by: §2 .

 X. Chen, J. Xu, T. Liang, Z. He, J. Pang, D. Yu, L. Song, Q. Liu, M. Zhou, Z. Zhang, et al. (2024) 
 Do not think that much for 2+ 3=? on the overthinking of o1-like llms .

 arXiv preprint arXiv:2412.21187 .

 Cited by: §5.5 .

 G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen, et al. (2025) 
 Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities .

 arXiv preprint arXiv:2507.06261 .

 Cited by: §1 .

 G. Cui, Y. Zhang, J. Chen, L. Yuan, Z. Wang, Y. Zuo, H. Li, Y. Fan, H. Chen, W. Chen, et al. (2025) 
 The entropy mechanism of reinforcement learning for reasoning language models .

 arXiv preprint arXiv:2505.22617 .

 Cited by: Figure 8 ,
 §H.1 .

 Y. Du, S. Zhao, Y. Gao, D. Zhao, Q. Lin, M. Ma, J. Li, Y. Jiang, K. He, Q. Xu, et al. (2026) 
 S3-cot: self-sampled succinct reasoning enables efficient chain-of-thought llms .

 arXiv preprint arXiv:2602.01982 .

 Cited by: §2 .

 L. Gao, J. Tow, S. Biderman, S. Black, A. DiPofi, C. Foster, L. Golding, J. Hsu, K. McDonell, N. Muennighoff, et al. (2021) 
 A framework for few-shot language model evaluation .

 Cited by: footnote 2 .

 Y. Gu, L. Dong, F. Wei, and M. Huang (2023) 
 Minillm: knowledge distillation of large language models .

 In arXiv preprint arXiv:2306.08543 ,

 Cited by: §5.5 .

 D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Bi, et al. (2025) 
 Deepseek-r1: incentivizing reasoning capability in llms via reinforcement learning .

 arXiv preprint arXiv:2501.12948 .

 Cited by: §1 ,
 §5.6 .

 D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt (2020) 
 Measuring massive multitask language understanding .

 arXiv preprint arXiv:2009.03300 .

 Cited by: Table 2 .

 D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt (2021) 
 Measuring mathematical problem solving with the math dataset .

 arXiv preprint arXiv:2103.03874 .

 Cited by: Appendix F ,
 §5.1 .

 B. Hou, Y. Zhang, J. Ji, Y. Liu, K. Qian, J. Andreas, and S. Chang (2025) 
 Thinkprune: pruning long chain-of-thought of llms via reinforcement learning .

 arXiv preprint arXiv:2504.01296 .

 Cited by: §2 .

 K. Huang, S. Liu, X. Hu, T. Xu, L. Bao, and X. Xia (2025) 
 Reasoning efficiently through adaptive chain-of-thought compression: a self-optimizing framework .

 arXiv preprint arXiv:2509.14093 .

 Cited by: Table 1 ,
 §1 ,
 §2 .

 J. Hübotter, F. Lübeck, L. Behric, A. Baumann, M. Bagatella, D. Marta, I. Hakimi, I. Shenfeld, T. K. Buening, C. Guestrin, et al. (2026) 
 Reinforcement learning via self-distillation .

 arXiv preprint arXiv:2601.20802 .

 Cited by: §2 .

 A. Jaech, A. Kalai, A. Lerer, A. Richardson, A. El-Kishky, A. Low, A. Helyar, A. Madry, A. Beutel, A. Carney, et al. (2024) 
 Openai o1 system card .

 arXiv preprint arXiv:2412.16720 .

 Cited by: §1 .

 L. Li, J. Hao, J. K. Liu, Z. Zhou, Y. Miao, W. Pang, X. Tan, W. Chu, Z. Wang, S. Pan, et al. (2025a) 
 The choice of divergence: a neglected key to mitigating diversity collapse in reinforcement learning with verifiable reward .

 arXiv preprint arXiv:2509.07430 .

 Cited by: §5.5 .

 Y. Li, L. Ma, J. Zhang, L. Tang, W. Zhang, and G. Luo (2025b) 
 Leash: adaptive length penalty and reward shaping for efficient large reasoning model .

 arXiv preprint arXiv:2512.21540 .

 Cited by: §2 .

 Y. Li, B. Bergner, Y. Zhao, V. P. Patil, B. Chen, and C. Wang (2026) 
 Steering large reasoning models towards concise reasoning via flow matching .

 arXiv preprint arXiv:2602.05539 .

 Cited by: §2 .

 W. Lin, X. Li, Z. Yang, X. Fu, H. Zhen, Y. Wang, X. Yu, W. Liu, X. Li, and M. Yuan (2025) 
 Trimr: verifier-based training-free thinking compression for efficient test-time scaling .

 arXiv preprint arXiv:2505.17155 .

 Cited by: §2 .

 S. Liu, X. Dong, X. Lu, S. Diao, M. Liu, M. Chen, H. Yin, Y. Wang, K. Cheng, Y. Choi, et al. (2025) 
 DLER: doing length penalty right – incentivizing more intelligence per token via reinforcement learning .

 arXiv preprint arXiv:2510.15110 .

 Cited by: Figure 8 ,
 §H.1 ,
 Table 1 ,
 §1 ,
 §2 .

 N. Muennighoff, Z. Yang, W. Shi, X. L. Li, L. Fei-Fei, H. Hajishirzi, L. Zettlemoyer, P. Liang, E. Candès, and T. B. Hashimoto (2025) 
 S1: simple test-time scaling .

 In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing ,

 pp. 20286–20332 .

 Cited by: §1 .

 I. Shenfeld, M. Damani, J. Hübotter, and P. Agrawal (2026) 
 Self-distillation enables continual learning .

 arXiv preprint arXiv:2601.19897 .

 Cited by: §A.2 ,
 Figure 3 ,
 Figure 5 ,
 §C.2 ,
 Appendix G ,
 §1 ,
 §2 ,
 §2 ,
 Remark 3 .

 G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu (2025) 
 Hybridflow: a flexible and efficient rlhf framework .

 In Proceedings of the Twentieth European Conference on Computer Systems ,

 pp. 1279–1297 .

 Cited by: Appendix F ,
 Appendix F ,
 footnote 1 .

 C. Snell, J. Lee, K. Xu, and A. Kumar (2024) 
 Scaling llm test-time compute optimally can be more effective than scaling model parameters .

 arXiv preprint arXiv:2408.03314 .

 Cited by: §1 .

 Q. Wan, Z. Xu, L. Wei, X. Shen, and J. Sun (2026) 
 Mitigating overthinking in large reasoning models via difficulty-aware reinforcement learning .

 arXiv preprint arXiv:2601.21418 .

 Cited by: Table 1 ,
 §1 ,
 §2 .

 C. Wang, Y. Feng, D. Chen, Z. Chu, R. Krishna, and T. Zhou (2025a) 
 Wait, we don’t need to" wait"! removing thinking tokens improves reasoning efficiency .

 arXiv preprint arXiv:2506.08343 .

 Cited by: §2 .

 S. Wang, L. Yu, C. Gao, C. Zheng, S. Liu, R. Lu, K. Dang, X. Chen, J. Yang, Z. Zhang, et al. (2025b) 
 Beyond the 80/20 rule: high-entropy minority tokens drive effective reinforcement learning for llm reasoning .

 arXiv preprint arXiv:2506.01939 .

 Cited by: §H.1 ,
 §5.4 .

 Y. Wu, J. Shi, B. Wu, J. Zhang, X. Lin, N. Tang, and Y. Luo (2025) 
 Concise reasoning, big gains: pruning long reasoning trace with difficulty-aware prompting .

 arXiv preprint arXiv:2505.19716 .

 Cited by: §2 .

 H. Xia, C. T. Leong, W. Wang, Y. Li, and W. Li (2025) 
 Tokenskip: controllable chain-of-thought compression in llms .

 Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing , pp. 3351–3363 .

 Cited by: §2 .

 S. Xu, W. Xie, L. Zhao, and P. He (2025) 
 Chain of draft: thinking faster by writing less .

 arXiv preprint arXiv:2502.18600 .

 Cited by: Table 1 ,
 §2 .

 Y. Xu, H. Sang, Z. Zhou, R. He, and Z. Wang (2026a) 
 Overconfident errors need stronger correction: asymmetric confidence penalties for reinforcement learning .

 arXiv preprint arXiv:2602.21420 .

 Cited by: §2 .

 Y. Xu, H. Sang, Z. Zhou, R. He, and Z. Wang (2026b) 
 PACED: distillation at the frontier of student competence .

 arXiv preprint arXiv:2603.11178 .

 Cited by: §2 .

 A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, et al. (2025) 
 Qwen3 technical report .

 arXiv preprint arXiv:2505.09388 .

 Cited by: §1 ,
 §5.1 .

 T. Ye, L. Dong, X. Wu, S. Huang, and F. Wei (2026) 
 On-policy context distillation for language models .

 arXiv preprint arXiv:2602.12275 .

 Cited by: Table 1 ,
 §2 .

 Q. Yu, Z. Zhang, R. Zhu, Y. Yuan, X. Zuo, Y. Yue, W. Dai, T. Fan, G. Liu, L. Liu, et al. (2025) 
 Dapo: an open-source llm reinforcement learning system at scale .

 arXiv preprint arXiv:2503.14476 .

 Cited by: Appendix F ,
 §5.1 .

 Y. Zhang, S. Jiang, R. Li, J. Tu, Y. Su, L. Deng, X. Guo, C. Lv, and J. Lin (2026) 
 DeepPlanning: benchmarking long-horizon agentic planning with verifiable constraints .

 arXiv preprint arXiv:2601.18137 .

 Cited by: Appendix J ,
 Appendix J .

 S. Zhao, Z. Xie, M. Liu, J. Huang, G. Pang, F. Chen, and A. Grover (2026) 
 Self-distilled reasoner: on-policy self-distillation for large language models .

 arXiv preprint arXiv:2601.18734 .

 Cited by: §2 ,
 §3.3 .

 L. Zheng, L. Yin, Z. Xie, C. L. Sun, J. Huang, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, et al. (2024) 
 Sglang: efficient execution of structured language model programs .

 Advances in neural information processing systems 37 , pp. 62557–62583 .

 Cited by: Appendix F .

 Appendix A Theoretical Analysis

We provide a formal analysis of CRISP’s key properties: accuracy preservation guarantees (Section  A.3 ), formalization of difficulty-adaptive compression (Section  A.4 ), forgetting bounds relative to the base model (Section  A.5 ), and a probabilistic model of how compression reduces compounding errors (Section  A.6 ). We begin with two standard preliminary results that underpin the later proofs (Section  A.1 – A.2 ).

 A.1 Preliminary: Sequence-Level Divergence

By the standard chain rule of KL divergence for autoregressive models, the per-token CRISP objective (Eq.  1 ) equals the sequence-level KL divergence between student and teacher.

 Lemma 1 (Chain rule of KL for autoregressive models) . 

 For autoregressive distributions q ​ ( y ∣ x ) = ∏ t q ​ ( y t ∣ x , y < t ) q(y\mid x)=\prod_{t}q(y_{t}\mid x,y_{<t}) and p ​ ( y ∣ x ) = ∏ t p ​ ( y t ∣ x , y < t ) p(y\mid x)=\prod_{t}p(y_{t}\mid x,y_{<t}) , the sequence-level KL decomposes as D KL ( q ∥ p ) = 𝔼 y ∼ q [ ∑ t D KL ( q ( ⋅ ∣ x , y < t ) ∥ p ( ⋅ ∣ x , y < t ) ) ] D_{\mathrm{KL}}(q\|p)=\mathbb{E}_{y\sim q}[\sum_{t}D_{\mathrm{KL}}(q(\cdot\mid x,y_{<t})\|p(\cdot\mid x,y_{<t}))] . This follows directly from expanding log ⁡ q ​ ( y ) / p ​ ( y ) = ∑ t log ⁡ q ​ ( y t ∣ ⋅ ) / p ​ ( y t ∣ ⋅ ) \log q(y)/p(y)=\sum_{t}\log q(y_{t}\mid\cdot)/p(y_{t}\mid\cdot) and applying the tower property. 

 Corollary 1 . 

 Identifying q = π θ ( ⋅ ∣ x ) q=\pi_{\theta}(\cdot\mid x) and p = π θ ¯ ( ⋅ ∣ x , c ) p=\pi_{\bar{\theta}}(\cdot\mid x,c) , the CRISP training loss equals the expected sequence-level KL: ℒ ( θ ) = 𝔼 x ∼ 𝒟 [ D KL ( π θ ( ⋅ ∣ x ) ∥ π θ ¯ ( ⋅ ∣ x , c ) ) ] \mathcal{L}(\theta)=\mathbb{E}_{x\sim\mathcal{D}}[D_{\mathrm{KL}}(\pi_{\theta}(\cdot\mid x)\|\pi_{\bar{\theta}}(\cdot\mid x,c))] . 

 A.2 Preliminary: Implicit Reward Interpretation

Following  Shenfeld et al. ( 2026 ) , the reverse-KL objective admits an implicit reward interpretation.

 Proposition  [Implicit reward, restated from § 4 ]. 
The CRISP objective (Eq.  1 ) is equivalent to maximizing the expected implicit reward r ​ ( y t , x ) = log ⁡ π θ ¯ ​ ( y t ∣ x , c , y < t ) − log ⁡ π θ ​ ( y t ∣ x , y < t ) r(y_{t},x)=\log\pi_{\bar{\theta}}(y_{t}\mid x,c,y_{<t})-\log\pi_{\theta}(y_{t}\mid x,y_{<t}) . 

Proof of the implicit reward identity.

Rewriting log ⁡ ( π θ / π θ ¯ ) = − r \log(\pi_{\theta}/\pi_{\bar{\theta}})=-r shows that the per-token reverse KL equals − 𝔼 y t ∼ π θ ​ [ r ​ ( y t , x ) ] -\mathbb{E}_{y_{t}\sim\pi_{\theta}}[r(y_{t},x)] . Summing over positions and applying Lemma  1 gives ℒ ​ ( θ ) = − 𝔼 x , y ∼ π θ ​ [ ∑ t r ​ ( y t , x ) ] \mathcal{L}(\theta)=-\mathbb{E}_{x,\,y\sim\pi_{\theta}}[\sum_{t}r(y_{t},x)] .
∎

The reward is positive on tokens the concise teacher prefers and negative on tokens the student overproduces, so CRISP suppresses verbosity without an explicit length penalty.

 A.3 Accuracy Preservation under Compression

We show that if self-distillation converges (the training loss is small) and the concise teacher preserves accuracy, then the student’s accuracy is guaranteed to remain close to the base model’s.

 Definition 1 (Accuracy) . 

For a problem distribution 𝒟 \mathcal{D} with correct-answer sets { A ​ ( x ) } x ∈ 𝒟 \{A(x)\}_{x\in\mathcal{D}} , the accuracy of policy π \pi is Acc ​ ( π ) = 𝔼 x ∼ 𝒟 ​ [ π ​ ( A ​ ( x ) ∣ x ) ] \mathrm{Acc}(\pi)=\mathbb{E}_{x\sim\mathcal{D}}\big[\pi(A(x)\mid x)\big] , where π ​ ( A ​ ( x ) ∣ x ) = ∑ y ∈ A ​ ( x ) π ​ ( y ∣ x ) \pi(A(x)\mid x)=\sum_{y\in A(x)}\pi(y\mid x) .

 Proposition 1 (Accuracy preservation) . 

 Let π θ ∗ \pi_{\theta^{*}} denote the converged student with training loss ℒ ​ ( θ ∗ ) ≤ ϵ KL \mathcal{L}(\theta^{*})\leq\epsilon_{\mathrm{KL}} . Suppose the concise teacher preserves accuracy relative to the base model: 

 Acc ( π θ ¯ ( ⋅ ∣ ⋅ , c ) ) ≥ Acc ( π θ ¯ ) − ϵ T . \mathrm{Acc}\big(\pi_{\bar{\theta}}(\cdot\mid\cdot,c)\big)\geq\mathrm{Acc}(\pi_{\bar{\theta}})-\epsilon_{T}. 

 (6) 

 Then the student satisfies: 

 Acc ​ ( π θ ∗ ) ≥ Acc ​ ( π θ ¯ ) − ϵ T − ϵ KL 2 . \mathrm{Acc}(\pi_{\theta^{*}})\geq\mathrm{Acc}(\pi_{\bar{\theta}})-\epsilon_{T}-\sqrt{\frac{\epsilon_{\mathrm{KL}}}{2}}. 

 (7) 

Proof.

By Corollary  1 , we have 𝔼 x ∼ 𝒟 [ D KL ( π θ ∗ ( ⋅ ∣ x ) ∥ π θ ¯ ( ⋅ ∣ x , c ) ) ] ≤ ϵ KL \mathbb{E}_{x\sim\mathcal{D}}\big[D_{\mathrm{KL}}(\pi_{\theta^{*}}(\cdot\mid x)\|\pi_{\bar{\theta}}(\cdot\mid x,c))\big]\leq\epsilon_{\mathrm{KL}} .

 Step 1: KL to total variation. For each problem x x , Pinsker’s inequality gives:

 d TV ( π θ ∗ ( ⋅ ∣ x ) , π θ ¯ ( ⋅ ∣ x , c ) ) ≤ 1 2 D KL ( π θ ∗ ( ⋅ ∣ x ) ∥ π θ ¯ ( ⋅ ∣ x , c ) ) . d_{\mathrm{TV}}\big(\pi_{\theta^{*}}(\cdot\mid x),\;\pi_{\bar{\theta}}(\cdot\mid x,c)\big)\leq\sqrt{\tfrac{1}{2}\,D_{\mathrm{KL}}\big(\pi_{\theta^{*}}(\cdot\mid x)\|\pi_{\bar{\theta}}(\cdot\mid x,c)\big)}. 

 (8) 

Taking expectations over x ∼ 𝒟 x\sim\mathcal{D} and applying Jensen’s inequality (using concavity of ⋅ \sqrt{\cdot} ):

 𝔼 x [ d TV ( π θ ∗ ( ⋅ ∣ x ) , π θ ¯ ( ⋅ ∣ x , c ) ) ] ≤ 1 2 𝔼 x [ D KL ( π θ ∗ ( ⋅ ∣ x ) ∥ π θ ¯ ( ⋅ ∣ x , c ) ) ] ≤ ϵ KL 2 . \mathbb{E}_{x}\big[d_{\mathrm{TV}}\big(\pi_{\theta^{*}}(\cdot\mid x),\;\pi_{\bar{\theta}}(\cdot\mid x,c)\big)\big]\leq\sqrt{\tfrac{1}{2}\,\mathbb{E}_{x}\!\big[D_{\mathrm{KL}}\big(\pi_{\theta^{*}}(\cdot\mid x)\|\pi_{\bar{\theta}}(\cdot\mid x,c)\big)\big]}\leq\sqrt{\tfrac{\epsilon_{\mathrm{KL}}}{2}}. 

 (9) 

 Step 2: Total variation to accuracy. Since total variation bounds the difference in probability of any event, in particular the correctness event A ​ ( x ) A(x) :

 | π θ ∗ ( A ( x ) ∣ x ) − π θ ¯ ( A ( x ) ∣ x , c ) | ≤ d TV ( π θ ∗ ( ⋅ ∣ x ) , π θ ¯ ( ⋅ ∣ x , c ) ) . \big|\pi_{\theta^{*}}(A(x)\mid x)-\pi_{\bar{\theta}}(A(x)\mid x,c)\big|\leq d_{\mathrm{TV}}\big(\pi_{\theta^{*}}(\cdot\mid x),\;\pi_{\bar{\theta}}(\cdot\mid x,c)\big). 

 (10) 

Taking expectations over x ∼ 𝒟 x\sim\mathcal{D} and using | 𝔼 ​ [ f ] | ≤ 𝔼 ​ [ | f | ] |\mathbb{E}[f]|\leq\mathbb{E}[|f|] :

 | Acc ( π θ ∗ ) − Acc ( π θ ¯ ( ⋅ ∣ ⋅ , c ) ) | ≤ 𝔼 x [ d TV ( π θ ∗ ( ⋅ ∣ x ) , π θ ¯ ( ⋅ ∣ x , c ) ) ] ≤ ϵ KL 2 . \big|\mathrm{Acc}(\pi_{\theta^{*}})-\mathrm{Acc}\big(\pi_{\bar{\theta}}(\cdot\mid\cdot,c)\big)\big|\leq\mathbb{E}_{x}\big[d_{\mathrm{TV}}\big(\pi_{\theta^{*}}(\cdot\mid x),\;\pi_{\bar{\theta}}(\cdot\mid x,c)\big)\big]\leq\sqrt{\tfrac{\epsilon_{\mathrm{KL}}}{2}}. 

 (11) 

 Step 3: Combine with teacher quality. From the assumption Acc ( π θ ¯ ( ⋅ ∣ ⋅ , c ) ) ≥ Acc ( π θ ¯ ) − ϵ T \mathrm{Acc}(\pi_{\bar{\theta}}(\cdot\mid\cdot,c))\geq\mathrm{Acc}(\pi_{\bar{\theta}})-\epsilon_{T} :

 Acc ​ ( π θ ∗ ) \displaystyle\mathrm{Acc}(\pi_{\theta^{*}}) 
 ≥ Acc ( π θ ¯ ( ⋅ ∣ ⋅ , c ) ) − ϵ KL 2 ≥ Acc ( π θ ¯ ) − ϵ T − ϵ KL 2 . ∎ \displaystyle\geq\mathrm{Acc}\big(\pi_{\bar{\theta}}(\cdot\mid\cdot,c)\big)-\sqrt{\tfrac{\epsilon_{\mathrm{KL}}}{2}}\geq\mathrm{Acc}(\pi_{\bar{\theta}})-\epsilon_{T}-\sqrt{\tfrac{\epsilon_{\mathrm{KL}}}{2}}.\qed 

 (12) 

 Remark 1 (When the bound is vacuous, and why that is informative) . 

 Proposition  1 identifies two independent sources of potential accuracy loss: the teacher accuracy gap ϵ T \epsilon_{T} and the student–teacher divergence ϵ KL / 2 \sqrt{\epsilon_{\mathrm{KL}}/2} . In our experiments, ϵ T \epsilon_{T} is consistently negative (the concise teacher is more accurate than the base model), making the bound vacuous in the traditional sense. This is informative rather than a weakness: it reveals why CRISP improves accuracy. The bound becomes Acc ​ ( π θ ∗ ) ≥ Acc ​ ( π θ ¯ ) + | ϵ T | − ϵ KL / 2 \mathrm{Acc}(\pi_{\theta^{*}})\geq\mathrm{Acc}(\pi_{\bar{\theta}})+|\epsilon_{T}|-\sqrt{\epsilon_{\mathrm{KL}}/2} , so accuracy improves whenever the teacher’s accuracy gain exceeds the distillation gap. The theorem is most useful in the regime of aggressive compression where ϵ T \epsilon_{T} might turn positive; it then quantifies the worst-case accuracy degradation. 

 A.4 Difficulty-Adaptive Compression

We formalize the empirical observation (Table  2 ) that CRISP compresses easy problems aggressively while preserving reasoning on hard problems.

 Definition 2 (Essential and compressible tokens) . 

For problem x x and student rollout y ∼ π θ ( ⋅ ∣ x ) y\sim\pi_{\theta}(\cdot\mid x) , classify each token position t t based on the implicit reward sign (Eq.  3 ):

 ℰ ​ ( x , y ) \displaystyle\mathcal{E}(x,y) 
 = { t : π θ ¯ ​ ( y t ∣ x , c , y < t ) ≥ π θ ​ ( y t ∣ x , y < t ) } \displaystyle=\big\{t:\pi_{\bar{\theta}}(y_{t}\mid x,c,y_{<t})\geq\pi_{\theta}(y_{t}\mid x,y_{<t})\big\} 
 (essential:  ​ r ​ ( y t , x ) ≥ 0 ​ ) , \displaystyle\text{(essential: }r(y_{t},x)\geq 0\text{)}, 

 (13) 

 𝒞 ​ ( x , y ) \displaystyle\mathcal{C}(x,y) 
 = { t : π θ ¯ ​ ( y t ∣ x , c , y < t ) < π θ ​ ( y t ∣ x , y < t ) } \displaystyle=\big\{t:\pi_{\bar{\theta}}(y_{t}\mid x,c,y_{<t})<\pi_{\theta}(y_{t}\mid x,y_{<t})\big\} 
 (compressible:  ​ r ​ ( y t , x ) < 0 ​ ) . \displaystyle\text{(compressible: }r(y_{t},x)<0\text{)}. 

 (14) 

 Proposition 2 (Difficulty-adaptive compression signal) . 

 Let d ​ ( x ) ∈ [ 0 , 1 ] d(x)\in[0,1] denote problem difficulty, defined as the base model’s failure rate d ​ ( x ) = 1 − π θ 0 ​ ( A ​ ( x ) ∣ x ) d(x)=1-\pi_{\theta_{0}}(A(x)\mid x) . Assume: 

 (A1) 

 Essential fraction increases with difficulty: The expected fraction of essential tokens ρ ​ ( x ) ≔ 𝔼 y ​ [ | ℰ ​ ( x , y ) | / | y | ] \rho(x)\coloneqq\mathbb{E}_{y}[|\mathcal{E}(x,y)|/|y|] is non-decreasing in d ​ ( x ) d(x) . 

 (A2) 

 Category-level KL is problem-independent: There exist constants D ℰ , D 𝒞 > 0 D_{\mathcal{E}},D_{\mathcal{C}}>0 such that for all problems x x : 

 𝔼 [ D KL ( π θ ( ⋅ ∣ x , y < t ) ∥ π θ ¯ ( ⋅ ∣ x , c , y < t ) ) | t ∈ 𝒞 ( x , y ) ] \displaystyle\mathbb{E}\big[D_{\mathrm{KL}}\big(\pi_{\theta}(\cdot\mid x,y_{<t})\|\pi_{\bar{\theta}}(\cdot\mid x,c,y_{<t})\big)\,\big|\,t\in\mathcal{C}(x,y)\big] 
 = D 𝒞 , \displaystyle=D_{\mathcal{C}}, 

 (15) 

 𝔼 [ D KL ( π θ ( ⋅ ∣ x , y < t ) ∥ π θ ¯ ( ⋅ ∣ x , c , y < t ) ) | t ∈ ℰ ( x , y ) ] \displaystyle\mathbb{E}\big[D_{\mathrm{KL}}\big(\pi_{\theta}(\cdot\mid x,y_{<t})\|\pi_{\bar{\theta}}(\cdot\mid x,c,y_{<t})\big)\,\big|\,t\in\mathcal{E}(x,y)\big] 
 = D ℰ . \displaystyle=D_{\mathcal{E}}. 

 (16) 

 (A3) 

 Compressible tokens carry strictly larger KL: D 𝒞 > D ℰ D_{\mathcal{C}}>D_{\mathcal{E}} . 

 Then the expected normalized compression signal 

 S ( x ) = 𝔼 y ∼ π θ ( ⋅ ∣ x ) [ 1 | y | ∑ t = 1 | y | D KL ( π θ ( ⋅ ∣ x , y < t ) ∥ π θ ¯ ( ⋅ ∣ x , c , y < t ) ) ] S(x)=\mathbb{E}_{y\sim\pi_{\theta}(\cdot\mid x)}\!\left[\frac{1}{|y|}\sum_{t=1}^{|y|}D_{\mathrm{KL}}\big(\pi_{\theta}(\cdot\mid x,y_{<t})\|\pi_{\bar{\theta}}(\cdot\mid x,c,y_{<t})\big)\right] 

 (17) 

 is non-increasing in d ​ ( x ) d(x) . 

Proof.

Decompose the normalized KL into essential and compressible contributions. For a given rollout y y :

 1 | y | ​ ∑ t D KL ​ ( q t ∥ p t ) = | ℰ | | y | ⋅ D ¯ ℰ ⏟ essential term + | 𝒞 | | y | ⋅ D ¯ 𝒞 ⏟ compressible term , \frac{1}{|y|}\sum_{t}D_{\mathrm{KL}}(q_{t}\|p_{t})=\underbrace{\frac{|\mathcal{E}|}{|y|}\cdot\bar{D}_{\mathcal{E}}}_{\text{essential term}}+\underbrace{\frac{|\mathcal{C}|}{|y|}\cdot\bar{D}_{\mathcal{C}}}_{\text{compressible term}}, 

 (18) 

where q t = π θ ( ⋅ ∣ x , y < t ) q_{t}=\pi_{\theta}(\cdot\mid x,y_{<t}) , p t = π θ ¯ ( ⋅ ∣ x , c , y < t ) p_{t}=\pi_{\bar{\theta}}(\cdot\mid x,c,y_{<t}) , and D ¯ ℰ , D ¯ 𝒞 \bar{D}_{\mathcal{E}},\bar{D}_{\mathcal{C}} denote the average per-token KL on essential and compressible tokens in this rollout, respectively. Taking expectations over y y and applying assumption (A2):

 S ​ ( x ) \displaystyle S(x) 
 = ρ ​ ( x ) ⋅ D ℰ + ( 1 − ρ ​ ( x ) ) ⋅ D 𝒞 \displaystyle=\rho(x)\cdot D_{\mathcal{E}}+(1-\rho(x))\cdot D_{\mathcal{C}} 

 (19) 

 = D 𝒞 − ρ ​ ( x ) ⋅ ( D 𝒞 − D ℰ ) . \displaystyle=D_{\mathcal{C}}-\rho(x)\cdot(D_{\mathcal{C}}-D_{\mathcal{E}}). 

 (20) 

By (A3), D 𝒞 − D ℰ > 0 D_{\mathcal{C}}-D_{\mathcal{E}}>0 , so S ​ ( x ) S(x) is a strictly decreasing affine function of ρ ​ ( x ) \rho(x) . Since ρ ​ ( x ) \rho(x) is non-decreasing in d ​ ( x ) d(x) by (A1), S ​ ( x ) S(x) is non-increasing in d ​ ( x ) d(x) .

 Quantitatively , for two problems with difficulties d 1 < d 2 d_{1}<d_{2} (hence ρ ​ ( x 1 ) ≤ ρ ​ ( x 2 ) \rho(x_{1})\leq\rho(x_{2}) by A1):

 S ​ ( x 1 ) − S ​ ( x 2 ) = ( ρ ​ ( x 2 ) − ρ ​ ( x 1 ) ) ⋅ ( D 𝒞 − D ℰ ) ≥ 0 . ∎ S(x_{1})-S(x_{2})=\big(\rho(x_{2})-\rho(x_{1})\big)\cdot\big(D_{\mathcal{C}}-D_{\mathcal{E}}\big)\geq 0.\qed 

 (21) 

 Remark 2 . 

 The core assumption is (A1): harder problems have a larger fraction of essential tokens. This is empirically supported by Table  2 (MATH-500: 57–59% compression vs. AIME 2025: ∼ {\sim} 35%). Assumption (A2) is a modeling simplification; the proposition should be interpreted as holding for category-averaged KL values. Assumption (A3) does not follow from the reward sign alone—it is a structural assumption motivated by the intuition that compressible positions, where teacher and student distributions diverge most, carry larger full-vocabulary KL. 

 A.5 Bounded Forgetting from the Base Model

A central advantage of on-policy self-distillation over off-policy SFT is controlled divergence from the original model. We formalize this through the conciseness gap .

 Definition 3 (Conciseness gap) . 

The conciseness gap of the base model π θ 0 \pi_{\theta_{0}} under instruction c c on input x x is

 γ ( x ) = d TV ( π θ 0 ( ⋅ ∣ x ) , π θ 0 ( ⋅ ∣ x , c ) ) . \gamma(x)=d_{\mathrm{TV}}\big(\pi_{\theta_{0}}(\cdot\mid x),\;\pi_{\theta_{0}}(\cdot\mid x,c)\big). 

 (22) 

 Proposition 3 (Bounded forgetting under on-policy self-distillation) . 

 Consider the first teacher window where θ ¯ = θ 0 \bar{\theta}=\theta_{0} (frozen teacher). If the converged CRISP loss satisfies ℒ ​ ( θ ∗ ) ≤ ϵ KL \mathcal{L}(\theta^{*})\leq\epsilon_{\mathrm{KL}} , then: 

 𝔼 x ∼ 𝒟 [ d TV ( π θ ∗ ( ⋅ ∣ x ) , π θ 0 ( ⋅ ∣ x ) ) ] ≤ ϵ KL 2 + 𝔼 x ∼ 𝒟 [ γ ( x ) ] . \mathbb{E}_{x\sim\mathcal{D}}\big[d_{\mathrm{TV}}\big(\pi_{\theta^{*}}(\cdot\mid x),\;\pi_{\theta_{0}}(\cdot\mid x)\big)\big]\leq\sqrt{\frac{\epsilon_{\mathrm{KL}}}{2}}+\mathbb{E}_{x\sim\mathcal{D}}[\gamma(x)]. 

 (23) 

 For subsequent windows with periodic teacher update, the same bound holds with θ 0 \theta_{0} replaced by the teacher weights θ ¯ \bar{\theta} at the start of that window. Moreover, γ ​ ( x ) \gamma(x) is difficulty-adaptive: for hard problems where the conciseness instruction has little effect, γ ​ ( x ) ≈ 0 \gamma(x)\approx 0 , so forgetting is minimal. 

Proof.

By the triangle inequality for total variation distance:

 d TV ( π θ ∗ ( ⋅ ∣ x ) , π θ 0 ( ⋅ ∣ x ) ) ≤ d TV ( π θ ∗ ( ⋅ ∣ x ) , π θ 0 ( ⋅ ∣ x , c ) ) ⏟ student–teacher gap + d TV ( π θ 0 ( ⋅ ∣ x , c ) , π θ 0 ( ⋅ ∣ x ) ) ⏟ γ ​ ( x ) . d_{\mathrm{TV}}\big(\pi_{\theta^{*}}(\cdot\mid x),\;\pi_{\theta_{0}}(\cdot\mid x)\big)\leq\underbrace{d_{\mathrm{TV}}\big(\pi_{\theta^{*}}(\cdot\mid x),\;\pi_{\theta_{0}}(\cdot\mid x,c)\big)}_{\text{student--teacher gap}}+\underbrace{d_{\mathrm{TV}}\big(\pi_{\theta_{0}}(\cdot\mid x,c),\;\pi_{\theta_{0}}(\cdot\mid x)\big)}_{\gamma(x)}. 

 (24) 

The student–teacher gap is bounded via Pinsker’s inequality. By Corollary  1 , 𝔼 x [ D KL ( π θ ∗ ( ⋅ ∣ x ) ∥ π θ 0 ( ⋅ ∣ x , c ) ) ] ≤ ϵ KL \mathbb{E}_{x}[D_{\mathrm{KL}}(\pi_{\theta^{*}}(\cdot\mid x)\|\pi_{\theta_{0}}(\cdot\mid x,c))]\leq\epsilon_{\mathrm{KL}} . Applying Pinsker to each x x and Jensen’s inequality over x ∼ 𝒟 x\sim\mathcal{D} :

 𝔼 x [ d TV ( π θ ∗ ( ⋅ ∣ x ) , π θ 0 ( ⋅ ∣ x , c ) ) ] ≤ ϵ KL 2 . \mathbb{E}_{x}\big[d_{\mathrm{TV}}\big(\pi_{\theta^{*}}(\cdot\mid x),\;\pi_{\theta_{0}}(\cdot\mid x,c)\big)\big]\leq\sqrt{\tfrac{\epsilon_{\mathrm{KL}}}{2}}. 

 (25) 

Taking expectations on both sides of the triangle inequality yields ( 23 ).

For the difficulty-adaptive claim: on hard problems, the conciseness instruction cannot substantially alter the output distribution because most reasoning steps are essential. Thus π θ 0 ( ⋅ ∣ x , c ) ≈ π θ 0 ( ⋅ ∣ x ) \pi_{\theta_{0}}(\cdot\mid x,c)\approx\pi_{\theta_{0}}(\cdot\mid x) , giving γ ​ ( x ) ≈ 0 \gamma(x)\approx 0 .
∎

 Remark 3 (Comparison with off-policy SFT) . 

 Standard off-policy SFT minimizes − 𝔼 ( x , y ) ∼ 𝒟 T ​ [ log ⁡ π θ ​ ( y ∣ x ) ] -\mathbb{E}_{(x,y)\sim\mathcal{D}_{T}}[\log\pi_{\theta}(y\mid x)] on a fixed teacher dataset 𝒟 T \mathcal{D}_{T} . The analogous forgetting decomposition is: 

 d TV ( π θ SFT ( ⋅ ∣ x ) , π θ 0 ( ⋅ ∣ x ) ) ≤ d TV ( π θ SFT ( ⋅ ∣ x ) , 𝒟 T ( ⋅ ∣ x ) ) + d TV ( 𝒟 T ( ⋅ ∣ x ) , π θ 0 ( ⋅ ∣ x ) ) . d_{\mathrm{TV}}\big(\pi_{\theta_{\mathrm{SFT}}}(\cdot\mid x),\;\pi_{\theta_{0}}(\cdot\mid x)\big)\leq d_{\mathrm{TV}}\big(\pi_{\theta_{\mathrm{SFT}}}(\cdot\mid x),\;\mathcal{D}_{T}(\cdot\mid x)\big)+d_{\mathrm{TV}}\big(\mathcal{D}_{T}(\cdot\mid x),\;\pi_{\theta_{0}}(\cdot\mid x)\big). 

 (26) 

 The second term measures the distribution mismatch between the teacher’s data and the base model’s outputs, which can be substantially larger than the conciseness gap γ ​ ( x ) \gamma(x) , particularly when the teacher generates qualitatively different reasoning styles. Crucially, off-policy SFT drives π θ SFT \pi_{\theta_{\mathrm{SFT}}} toward 𝒟 T \mathcal{D}_{T} directly, while on-policy distillation generates data from the student’s own evolving distribution, inherently limiting divergence from the base model  (Shenfeld et al. , 2026 ) . 

 A.6 Compression Reduces Compounding Error

We provide a simple probabilistic model explaining why shorter reasoning traces can improve accuracy.

 Proposition 4 (Shorter traces reduce error accumulation) . 

 Suppose each token position independently introduces a reasoning error with probability p err ∈ ( 0 , 1 ) p_{\mathrm{err}}\in(0,1) , so a trace of length L L is correct with probability Acc ​ ( L ) = ( 1 − p err ) L \mathrm{Acc}(L)=(1-p_{\mathrm{err}})^{L} . Compressing from L L to α ​ L \alpha L tokens ( α ∈ ( 0 , 1 ) \alpha\in(0,1) ) without increasing the per-token error rate yields an accuracy ratio 

 Acc ​ ( α ​ L ) Acc ​ ( L ) = ( 1 − p err ) − ( 1 − α ) ​ L ≥ 1 + ( 1 − α ) ​ L ⋅ p err , \frac{\mathrm{Acc}(\alpha L)}{\mathrm{Acc}(L)}=(1-p_{\mathrm{err}})^{-(1-\alpha)L}\geq 1+(1-\alpha)L\cdot p_{\mathrm{err}}, 

 (27) 

 which grows exponentially in the number of removed tokens. The lower bound follows from − ln ⁡ ( 1 − p ) ≥ p -\ln(1-p)\geq p and e u ≥ 1 + u e^{u}\geq 1+u . 

Quantitative interpretation.

On MATH-500, CRISP removes ( 1 − α ) ​ L ≈ 2 , 750 (1{-}\alpha)L\approx 2{,}750 tokens ( α ≈ 0.41 \alpha\approx 0.41 , L ≈ 4 , 660 L\approx 4{,}660 ). With p err = 10 − 4 p_{\mathrm{err}}=10^{-4} , the accuracy ratio is ≥ 1.275 \geq 1.275 (linear bound) or ≈ e 0.275 ≈ 1.32 \approx e^{0.275}\approx 1.32 (exponential). The independence assumption is conservative: correlated errors (where one mistake derails subsequent steps) make compression even more beneficial, which helps explain why empirical gains (e.g., 70.0 → 86.1 70.0{\to}86.1 on MATH-500) exceed this simple model’s predictions.

 Appendix B Prompt Templates

Figure  3 shows the exact student and teacher prompt templates used in our default setup. Both policies share the same underlying model; the only difference is that the teacher receives a prepended conciseness instruction and no privileged supervision such as a ground-truth answer.

 Student Prompt   π θ ( ⋅ ∣ x ) \pi_{\theta}(\cdot\mid x) 

 Solve the following math problem step by step. The last line of your response should be of the form Answer: $Answer (without quotes) where $Answer is the answer to the problem.

Find all real numbers x x such that x 3 − 6 ​ x 2 + 11 ​ x − 6 = 0 x^{3}-6x^{2}+11x-6=0 .

Remember to put your answer on its own line after ‘‘Answer:’’. 

 Teacher Prompt   π θ ¯ ( ⋅ ∣ x , c ) \pi_{\bar{\theta}}(\cdot\mid x{,}\;c) 

 Conciseness instruction c c : Solve the following math problem concisely and correctly. Be direct --- avoid unnecessary elaboration, redundant steps, or restating the problem. Focus only on the key reasoning steps needed to reach the answer.

The last line of your response should be of the form Answer: $Answer (without quotes) where $Answer is the answer to the problem.

Find all real numbers x x such that x 3 − 6 ​ x 2 + 11 ​ x − 6 = 0 x^{3}-6x^{2}+11x-6=0 .

Remember to put your answer on its own line after ‘‘Answer:’’. 

 Figure 3: Prompt example for student and teacher policies. Both policies share the same model parameters but differ in conditioning context. The teacher receives only a conciseness instruction c c prepended to the problem; no ground-truth answers or reference solutions are provided. This is the key distinction from prior self-distillation work  (Shenfeld et al. , 2026 ) , where the teacher receives the ground-truth solution as privileged information. The student prompt is the original prompt from the DAPO-17K dataset. 

 Appendix C Ablation Study

 C.1 Do Quantitative Reduction Targets Outperform Qualitative Instructions?

Our default teacher simply says “be concise.” But what if we gave it a number? Telling the model to “use 50% fewer tokens” seems more precise—surely that would compress harder? We investigate this by comparing four context variants, all trained with the same periodic teacher update ( M = 50 M{=}50 ): the static conciseness instruction and soft budget targets at p ∈ { 20 % , 50 % , 80 % } p\in\{20\%,50\%,80\%\} . The soft budget teacher prompt (Figure  4 ) replaces the qualitative conciseness instruction with a specific reduction target while keeping all other aspects identical.

 Soft Budget Teacher Prompt   π θ ¯ ( ⋅ ∣ x , c p ) \pi_{\bar{\theta}}(\cdot\mid x{,}\;c_{p}) 

 Soft budget instruction c p c_{p} : Solve the following math problem correctly using p p % fewer tokens than you normally would. Be more concise --- cut unnecessary elaboration, redundant steps, and verbose explanations while preserving correctness.

The last line of your response should be of the form Answer: $Answer (without quotes) where $Answer is the answer to the problem.

Find all real numbers x x such that x 3 − 6 ​ x 2 + 11 ​ x − 6 = 0 x^{3}-6x^{2}+11x-6=0 .

Remember to put your answer on its own line after ‘‘Answer:’’. 

 Figure 4: Soft budget teacher prompt. Unlike the qualitative conciseness instruction in Figure  3 , the soft budget variant specifies a quantitative reduction target p ∈ { 20 , 50 , 80 } p\in\{20,50,80\} . The student prompt remains unchanged from Figure  3 . 

 Table 4: Qualitative conciseness instructions outperform quantitative reduction targets on accuracy (30K token budget). All variants use periodic teacher update ( M = 50 M{=}50 ) at step 100. Soft budgets achieve higher compression but substantially lower accuracy than the concise instruction, particularly on competition-level benchmarks. Accuracy (Acc, %), token reduction (Red., %), and accuracy change vs. base model ( Δ \Delta Acc, pp). 

 MATH-500 
 AIME 2024 
 AIME 2025 

 Context 
 Acc 
 Red. 

 Δ \Delta Acc 
 Acc 
 Red. 

 Δ \Delta Acc 
 Acc 
 Red. 

 Δ \Delta Acc 

 Qwen3-8B 

 Concise 
 86.6 
 58.8% 
 +8.9 
 69.6 
 35.4% 

 − - 2.9 
 57.1 
 35.7% 

 − - 5.4 

 Soft ( p = 20 % p{=}20\% ) 
 86.2 
 60.1% 
 +8.6 
 70.8 
 39.5% 

 − - 1.7 

 52.9 
 33.7% 

 − - 9.6 

 Soft ( p = 50 % p{=}50\% ) 
 85.9 
 60.4% 
 +8.2 
 63.8 
 36.3% 

 − - 8.8 
 52.1 
 36.2% 

 − - 10.4 

 Soft ( p = 80 % p{=}80\% ) 
 84.1 
 63.8% 
 +6.5 
 67.9 
 41.5% 

 − - 4.6 
 49.6 
 39.5% 

 − - 12.9 

 Qwen3-14B 

 Concise 
 86.1 
 56.5% 
 +16.1 
 76.3 
 41.0% 
 +10.5 
 61.7 
 35.2% 

 − - 5.4 

 Soft ( p = 20 % p{=}20\% ) 
 80.7 
 67.8% 
 +10.8 
 67.1 
 47.8% 
 +1.3 
 57.9 
 45.8% 

 − - 9.2 

 Soft ( p = 50 % p{=}50\% ) 
 80.7 
 67.2% 
 +10.8 
 67.1 
 49.7% 
 +1.3 
 57.5 
 48.8% 

 − - 9.6 

 Soft ( p = 80 % p{=}80\% ) 
 79.8 
 68.9% 
 +9.9 
 68.3 
 50.8% 
 +2.5 
 54.6 
 50.7% 

 − - 12.5 

Table  4 reveals a clear compression–accuracy tradeoff across context variants : soft budgets achieve higher compression but substantially lower accuracy than the qualitative concise instruction.

 Soft budgets compress more aggressively but sacrifice accuracy. On MATH-500, p = 80 % p{=}80\% achieves 63.8% token reduction for Qwen3-8B (versus the concise instruction’s 58.8%), but accuracy drops from 86.6% to 84.1%. The gap widens dramatically on competition-level benchmarks: for Qwen3-14B on AIME 2024, the concise instruction achieves 76.3% accuracy while all soft budget variants cluster around 67–68%. The concise instruction achieves the best accuracy on 5 of 6 model–benchmark combinations. The sole exception is Qwen3-8B on AIME 2024, where p = 20 % p{=}20\% reaches 70.8% versus 69.6% for the concise instruction, a difference within evaluation noise.

 Compression monotonically increases with p p , but accuracy does not. For Qwen3-14B, token reduction on AIME 2024 increases with the target: p = 20 % p{=}20\% achieves 47.8%, p = 50 % p{=}50\% achieves 49.7%, and p = 80 % p{=}80\% achieves 50.8%. However, the best soft-budget accuracy on AIME 2024 comes from p = 80 % p{=}80\% (68.3%), not p = 20 % p{=}20\% (67.1%), suggesting that the relationship between reduction target and accuracy is non-monotonic.

These results recommend the qualitative concise instruction as the default : it achieves the best accuracy, compresses substantially (57–59% on MATH-500), and—crucially—remains stable under extended training. The lesson: vague instructions make better teachers than precise ones.

 C.2 How Sensitive Is Compression to the Teacher Update Interval?

The teacher update interval M M (Eq.  2 ) controls how frequently the teacher weights are synchronized with the student. A larger M M provides a more stable distillation target but limits progressive compression; a smaller M M pushes compression further but risks training instability when the teacher changes too rapidly. We sweep M ∈ { 1 , 10 , 20 , 40 , 50 , 60 } M\in\{1,10,20,40,50,60\} using Qwen3-14B with the qualitative concise instruction on MATH-500.

 Figure 5: Teacher update interval M M controls the stability–compression trade-off. Accuracy (left) and output entropy (right) over 100 training steps for Qwen3-14B on MATH-500 with varying M M . M = 1 M{=}1 (updating every step) causes entropy explosion and accuracy collapse to ∼ {\sim} 2% by step 100, consistent with the instability observed by Shenfeld et al. ( 2026 ) . M ∈ { 40 , 50 , 60 } M\in\{40,50,60\} produce stable trajectories reaching ∼ {\sim} 86–87% accuracy. M = 10 M{=}10 peaks early then degrades, while M = 20 M{=}20 remains competitive but shows mild entropy drift. All experiments use the qualitative concise instruction. 

Figure  5 reveals three distinct regimes:

 M = 1 M{=}1 is catastrophically unstable. Updating the teacher after every gradient step causes entropy to explode from ∼ {\sim} 0.32 to ∼ {\sim} 0.58 and accuracy to collapse from a peak of ∼ {\sim} 82% (step 10) to ∼ {\sim} 2% (step 100). This mirrors the finding of Shenfeld et al. ( 2026 ) that overly aggressive teacher updates create a moving target problem: the student chases a teacher that is itself changing in response to the student’s updates, leading to a positive feedback loop of increasingly degenerate outputs.

 M ∈ { 40 , 50 , 60 } M\in\{40,50,60\} form a stable plateau. These intervals produce similar accuracy trajectories, all reaching ∼ {\sim} 86–87% by step 100 with entropy remaining stable around ∼ {\sim} 0.33–0.39. The method is robust to the exact choice of M M within this range: the teacher remains stable long enough for the student to meaningfully converge toward the current target before the next refresh.

 M = 10 M{=}10 degrades; M = 20 M{=}20 is borderline. M = 10 M{=}10 peaks at ∼ {\sim} 83% accuracy around step 50 but declines to ∼ {\sim} 80% by step 100, with entropy drifting up to ∼ {\sim} 0.44. M = 20 M{=}20 performs better (84.5% at step 100) but still trails the M ≥ 40 M\geq 40 regime by 2–3 percentage points.

Based on these results, we use M = 50 M{=}50 for all other experiments in this paper, as it sits comfortably in the stable plateau while allowing progressive compression through periodic teacher refresh.

 Appendix D Survey of Reasoning Compression Methods

Table  5 summarizes 19 reasoning compression methods along four axes. This survey motivates the design of CRISP by revealing the pervasive dependence on ground-truth answers and the rarity of difficulty-adaptive methods.

 Table 5: Survey of 19 reasoning compression methods. LP = Length Penalty in reward; DD = Difficulty-Dependent; CA = Correct Answer required; HB = Hard Budget. 

 Method 
 LP 
 DD 
 CA 
 HB 
 Approach 

 L1 
 ✓ 

 ✓ 
 ✓ 
 RL 

 DiPO 
 ✓ 
 ✓ 
 ✓ 

 RL 

 TRAAC 
 ✓ 
 ✓ 
 ✓ 

 RL 

 DIET 
 ✓ 
 ✓ 
 ✓ 

 RL 

 DLER 
 ✓ 
 ✓ 
 ✓ 

 RL 

 Leash 
 ✓ 

 ✓ 

 RL 

 ORION 
 ✓ 

 ✓ 

 RL 

 AdaptThink 

 ✓ 
 ✓ 

 RL 

 SEER 

 ✓ 

 SFT 

 TokenSkip 

 ✓ 
 ✓ 
 SFT 

 V-Skip 

 ✓ 

 SFT 

 S3-CoT 

 Steering 

 DAP/LiteCoT 

 ✓ 
 ✓ 

 SFT 

 Extra-CoT 

 ✓ 

 SFT 

 CtrlCoT 

 ✓ 
 ✓ 
 SFT 

 Chain of Draft 

 Prompt 

 TrimR 

 Inference 

 NoWait 

 Inference 

 FlowSteer 

 Inference 

 CRISP (Ours) 

 ✓ 

 Self-distill 

 Appendix E Extended Results Across Token Budgets

 E.1 Results Under 8K Token Budget

Table  6 reports results under the efficient-serving token budget of 8,192 tokens. Because the base model frequently produces responses exceeding this limit on harder benchmarks (AIME 2024/2025), truncation disproportionately affects the verbose baseline, making the compression gains appear even larger. We include these results as a practical reference for deployment scenarios where strict token budgets are enforced.

Truncation explains the amplified accuracy gains.

Comparing Tables  2 and  6 reveals a striking pattern: the accuracy gap between CRISP and the base model is far larger under the 8K budget than under 30K. For example, Qwen3-8B on AIME 2024 shows + 29.6 +29.6  pp under 8K ( 25.0 → 54.6 25.0{\to}54.6 ) but − 2.9 -2.9  pp under 30K ( 72.5 → 69.6 72.5{\to}69.6 ). The mechanism is straightforward: under the 8K cap, the verbose base model’s responses are frequently truncated before reaching the answer, causing catastrophic accuracy loss (the base model drops from 72.5% to 25.0% on AIME 2024 when the budget shrinks from 30K to 8K). CRISP’s compressed responses largely fit within the 8K window, avoiding this truncation penalty entirely. The lesson is practical: under strict serving budgets, reasoning compression is not merely an efficiency tool—it is essential for preserving accuracy .

 Table 6: Self-distillation results under the 8,192-token budget. Same setup as Table  2 but with max response length capped at 8,192 tokens, representative of efficient serving constraints. Accuracy (Acc, mean over 8 samples, %), average reasoning token length (Len), and token reduction (Red., %). 

 MATH-500 
 AIME 2024 
 AIME 2025 

 Method 
 Acc 
 Len 
 Red. 
 Acc 
 Len 
 Red. 
 Acc 
 Len 
 Red. 

 Qwen3-8B 

 Base Model 
 69.4 
 3,860 
 — 
 25.0 
 7,597 
 — 
 18.8 
 7,661 
 — 

 Concise prompt 
 78.1 
 2,514 
 34.9% 
 37.9 
 6,870 
 9.6% 
 29.2 
 7,140 
 6.8% 

 CRISP 
 85.1 
 1,199 
 68.9% 
 54.6 
 5,114 
 32.7% 
 36.3 
 5,658 
 26.2% 

 Qwen3-14B 

 Base Model 
 64.3 
 3,285 
 — 
 27.1 
 7,345 
 — 
 19.6 
 7,471 
 — 

 Concise prompt 
 81.8 
 2,059 
 37.3% 
 45.4 
 6,483 
 11.7% 
 30.8 
 6,852 
 8.3% 

 CRISP 
 85.5 
 1,066 
 67.6% 
 57.5 
 4,599 
 37.4% 
 39.2 
 5,484 
 26.6% 

 E.2 Extended Training Under 30K Token Budget

Table  7 shows performance at training steps 100 and 200 under the 30K token budget, illustrating the accuracy–compression trade-off as training progresses beyond the sweet spot. At step 100 (our default checkpoint), both models achieve strong compression with accuracy improvements on MATH-500. Continuing to step 200 roughly doubles token reduction on AIME benchmarks (from ∼ 35 {\sim}35 – 41 % 41\% to ∼ 51 {\sim}51 – 53 % 53\% ) but at the cost of AIME accuracy, particularly for the harder AIME 2025. MATH-500 accuracy remains robust throughout, suggesting that compression on easier benchmarks is more sustainable.

Practical recommendation: step 100 is the sweet spot.

Based on these results, we recommend step 100 (roughly 3,200 training examples) as the default checkpoint. It achieves 57–59% compression on MATH-500 with accuracy gains of 9–16 pp, while AIME accuracy degrades by at most 5 pp. Continuing to step 200 yields diminishing compression returns on MATH-500 (+13 pp additional reduction for Qwen3-8B) but substantial accuracy losses on competition benchmarks ( − 7.5 -7.5  pp on AIME 2024, − 10.4 -10.4  pp on AIME 2025 for 8B).

Difficulty-dependent over-compression.

The divergent behavior across benchmarks at step 200 is consistent with the difficulty-adaptive compression analysis in Section  A.4 . On easy benchmarks like MATH-500, most tokens are compressible (the essential fraction ρ ​ ( x ) \rho(x) is small), so further compression removes genuinely redundant tokens and accuracy is preserved. On hard benchmarks like AIME 2025, ρ ​ ( x ) \rho(x) is already high at step 100; continued training pushes compression into essential tokens, degrading accuracy. This provides a practical corollary to Proposition  2 : the optimal training duration is itself difficulty-dependent , and a single checkpoint must trade off easy-problem compression against hard-problem preservation.

 Table 7: Extended training results under the 30K token budget. Performance at step 100 (default) and step 200, showing the accuracy–compression trade-off with continued training. Accuracy (Acc, mean over 8 samples, %), average reasoning token length (Len), and token reduction vs. base model (Red., %). 

 MATH-500 
 AIME 2024 
 AIME 2025 

 Method 
 Acc 
 Len 
 Red. 
 Acc 
 Len 
 Red. 
 Acc 
 Len 
 Red. 

 Qwen3-8B 

 Base Model 
 77.7 
 4,661 
 — 
 72.5 
 14,170 
 — 
 62.5 
 16,682 
 — 

 CRISP (step 100) 
 86.6 
 1,921 
 58.8% 
 69.6 
 9,152 
 35.4% 
 57.1 
 10,726 
 35.7% 

 CRISP (step 200) 
 85.1 
 1,305 
 72.0% 
 62.1 
 6,902 
 51.3% 
 46.7 
 8,146 
 51.2% 

 Qwen3-14B 

 Base Model 
 70.0 
 3,872 
 — 
 65.8 
 12,844 
 — 
 67.1 
 15,642 
 — 

 CRISP (step 100) 
 86.1 
 1,686 
 56.5% 
 76.3 
 7,577 
 41.0% 
 61.7 
 10,137 
 35.2% 

 CRISP (step 200) 
 86.2 
 1,191 
 69.2% 
 66.3 
 6,089 
 52.6% 
 53.8 
 7,496 
 52.1% 

 Appendix F Training and Implementation Details

Technical setup.

All experiments are conducted on a single node equipped with eight NVIDIA H200 GPUs.
Our implementation is built on top of the verl library  Sheng et al. ( 2025 ) , which provides a HybridEngine for efficient actor–rollout–reference model co-location.
We use PyTorch Fully Sharded Data Parallel (FSDP) for distributed training with parameter and optimizer offloading to CPU, and SGLang  Zheng et al. ( 2024 ) for batched rollout generation.
Sequence parallelism (Ulysses, degree 4) is enabled during training to handle long sequences efficiently, while tensor parallelism (degree 2) is used for inference.
Mixed-precision training is performed in bfloat16 , and gradient checkpointing is enabled to reduce peak memory usage.

Training data.

Our training data is derived from DAPO-Math-17k  Yu et al. ( 2025 ) , a deduplicated set of ∼ 17 , 000 {\sim}17{,}000 competition-level math problems.
We randomly split the dataset into 80% training ( ∼ 13 , 600 {\sim}13{,}600 prompts) and 20% validation ( ∼ 3 , 400 {\sim}3{,}400 prompts) with a fixed seed for reproducibility across all configurations.
For each problem, we construct a student prompt (the original question) and a teacher prompt (the question prepended with a conciseness instruction).

Training procedure.

At each training step, the student model generates a response from the student prompt via SGLang sampling (temperature 1.0, top- p p 1.0).
We then perform a single gradient update minimizing the reverse KL divergence between student and teacher logit distributions over the student’s own generated tokens.
All student rollouts are used for training regardless of correctness; no filtering is applied.
Both teacher and student forward passes are performed for each micro-batch with chunked logit processing (chunk size 256 tokens) to bound peak GPU memory; teacher logits are progressively freed after each chunk.

Hyperparameters.

Table  8 summarizes the full configuration, which is shared across all models and instruction variants.