Title: 2603.00578 ArXiv: 2603.00578 Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs Title: Content selection saved. Describe the issue below: Description: License: CC BY-NC-ND 4.0 arXiv:2603.00578v1 [cs.AI] 28 Feb 2026 Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs Jie Cao 1 † † thanks: These authors contributed equally to this work. , Tianwei Lin 1 1 1 footnotemark: 1 , Zhenxuan Fan 1 , Bo Yuan 1 , Ziyuan Zhao 2 , Rolan Yan 2 , Wenqiao Zhang † † thanks: Corresponding author 1 , Siliang Tang 1 1 Zhejiang University, 2 Tencent Abstract Long chain-of-thought (CoT) has become a dominant paradigm for enhancing the reasoning capability of large reasoning models (LRMs); however, the performance gains often come with a substantial increase in reasoning budget. Recent studies show that existing CoT paradigms tend to induce systematic overthinking, unnecessarily coupling reasoning capability with reasoning cost. Most prior approaches reduce token usage through post hoc techniques such as token compression, truncation, or length penalties, without explicitly addressing the core mechanisms of reasoning. We propose Draft-Thinking , which guides models to first learn a concise draft-style reasoning structure that retains only the critical reasoning steps. Through a progressive curriculum learning , the model stably internalizes this efficient reasoning pattern as its capability scales. Moreover, Draft-Thinking introduces adaptive prompting, which elevates reasoning depth to a flexible, model-selectable behavior. Extensive experiments demonstrate that Draft-Thinking substantially reduces reasoning budget while largely preserving reasoning performance; for example, on MATH500, it achieves an 82.6% reduction in reasoning budget at the cost of only a 2.6% performance drop. Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs Jie Cao 1 † † thanks: These authors contributed equally to this work. , Tianwei Lin 1 1 1 footnotemark: 1 , Zhenxuan Fan 1 , Bo Yuan 1 , Ziyuan Zhao 2 , Rolan Yan 2 , Wenqiao Zhang † † thanks: Corresponding author 1 , Siliang Tang 1 1 Zhejiang University, 2 Tencent 1 Introduction Long chain-of-thought (CoT) combined with reinforcement learning becomes a mainstream pathway for reasoning-oriented large reasoning models (LRMs) to achieve high performance Comanici et al. ( 2025 ); OpenAI ( 2025 ) , as exemplified by DeepSeek-R1 Guo et al. ( 2025 ) and Qwen3 Yang et al. ( 2025a ) . However, long reasoning trajectories often introduce more stable rewards, implicitly binding reasoning correctness to reasoning length and inducing a behavioral bias toward high-budget reasoning. This phenomenon exposes a systematic issue: extensive reasoning expansion is not necessary at the task level and even degrades performance, giving rise to the notorious overthinking problem Chen et al. ( 2025b ); Cuadron et al. ( 2025 ) . To improve the reasoning efficiency of LRMs, existing approaches mainly fall into three optimization pathways. First, inspired by human cognitive science, the chain-of-draft (CoD) paradigm guides models via prompting to generate only key reasoning steps Xu et al. ( 2025 ); Aytes et al. ( 2025 ) . However, such methods heavily rely on prompt quality and exhibit substantial performance degradation in zero-shot settings and on small- to medium-scale models Xu et al. ( 2025 ) . Second, compression-based methods aim to reduce the reasoning budget by selecting and retaining high-value tokens or steps Xia et al. ( 2025 ); Yuan et al. ( 2025 ) . However, active compression depends on external importance estimation, making reasoning capability directly contingent on the quality of the estimator. Third, training methods that introduce length penalties significantly reduce token usage Hou et al. ( 2025 ); Luo et al. ( 2025 ); Song and Zheng ( 2025 ) . Nevertheless, they implicitly encourage a “longer-is-more-penalized” optimization objective, which causes systematic degradation in sampling quality as reasoning depth increases and collapses the task- and difficulty-specific reasoning depth requirements into a uniform low-budget preference. Figure 1: Illustration of the student learning process. Students distill core knowledge from teachers into concise drafts, progressively refine it through practice, and ultimately achieve mastery. Figure 2: Accuracy and token count comparison on the MATH500 dataset. Draft-Thinking achieves comparable or better accuracy with a smaller budget. To address the above limitations, we propose Draft-Thinking , which treats efficient reasoning as a learnable form and a decidable budget, rather than binding models to uniformly high-budget trajectories. This formulation enables flexible control of reasoning depth to adapt to problem complexity. Specifically, Draft-Thinking first internalizes the notion of Draft required for efficient reasoning, defining a reasoning cognition pattern constrained to “retaining only the key inferences that determine correctness.” In this way, the selection of high-value steps becomes an endogenous capability of the reasoning model, substantially reducing the number of explicit reasoning steps required. Building on this capability anchor, Draft-Thinking exposes the model to reasoning behaviors with varying degrees of expansion. Reasoning depth thus shifts from a fixed, prompt-driven generation habit to a behavior variable that the model can self-schedule. As a result, the model switches between low-budget and more exhaustive reasoning without external routing or explicit difficulty annotations, thereby avoiding worst-case uniform expansion under heterogeneous task difficulty. To stably acquire the above efficient reasoning and scheduling capabilities, we draw intuitive motivation from the student learning process: to develop concise and reliable problem-solving strategies, students typically first master core concepts and inferences and then internalize them into stable problem-solving cognition through repeated practice. Based on this insight, we propose a Progressive Curriculum Learning as illustrated in Figure 1 . The model first fits draft-style reasoning structures at the behavioral level through supervised fine-tuning (SFT), and leverages black-box distillation to extract implicit structured inference patterns from stronger models, establishing a reliable low-budget reasoning framework. It then introduces a two-stage reinforcement learning (RL) process to gradually expand problem-solving capability, stably learning hard tasks that require deep reasoning while maintaining a preference for low-budget efficiency. This training procedure does not rely on predefined compression ratios, templates, or length penalties; instead, it encourages the model to internalize the essential structure of concise reasoning through optimization. Draft-Thinking does not weaken the original long CoT capability. On the contrary, constraining the reasoning structure improves stability in long reasoning regimes. Moreover, built upon existing long CoT abilities, the method activates and amplifies efficient reasoning with only modest data and training overhead, making it a paradigm that can be invoked at low cost. Extensive experiments demonstrate the strong performance of Draft-Thinking. As shown in Figure 2 , the draft reasoning mode reduces the average token budget from 5,668 to 986 while maintaining an accuracy of 90.6%, achieving superior token efficiency. This improvement is consistently observed across different model scales and tasks. In summary, our key contributions are: (i) We propose Draft-Thinking , which internalizes the selection of high-value steps as an endogenous capability and reasoning pattern of LRMs, significantly outperforming existing methods in reasoning efficiency and flexibility. (ii) We elevate reasoning depth to a decidable dimension, enabling the reasoning budget to be flexibly scheduled according to task complexity. (iii) A Progressive Curriculum Learning strategy is introduced to integrate SFT with staged RL training, stably achieving the co-evolution of efficient reasoning structures and deep reasoning capability. 2 Related Work Recent research has revealed that LRMs suffer from an overthinking issue (Chen et al. , 2025b ; Cuadron et al. , 2025 ; Yang et al. , 2025b ) , where models with long CoT reasoning often expend unnecessary computational resources on redundant solutions that contribute minimally to final outcomes and may even degrade model performance. Prompt-based Efficient Reasoning. Prompt-based techniques (Xu et al. , 2025 ; Aytes et al. , 2025 ) improve reasoning efficiency through well-designed prompts with a few-shot setting. Chain-of-Draft (CoD) Xu et al. ( 2025 ) utilizes the simple instruction "Think step by step, but only keep a minimum draft for each thinking step, with 5 words at most." significantly reduces token usage for GPT-4o and Claude 3.5 Sonnet. Sketch-of-Thought Aytes et al. ( 2025 ) routes each instance to the most appropriate template among three cognitive science-inspired prompts. Compression of Chain-of-Thought. C3oT Kang et al. ( 2024 ) employs GPT-4 as a compressor to distill longer CoTs into shorter versions while preserving key information. It then trains LLMs on both longer and shorter CoT to learn their relationships and finally performs inference using the shorter CoT to achieve efficient reasoning. TokenSkip Xia et al. ( 2025 ) and CTS Yuan et al. ( 2025 ) compress original CoTs into shorter versions by applying different compression ratios according to token semantic importance, and then perform inference under specific compression ratios to achieve efficient reasoning. TALE Han et al. ( 2025 ) propose a token-budget-aware LLM reasoning framework that adjusts the number of reasoning tokens based on the complexity of each problem. Length-Regularized Reinforcement Learning. THINKPRUNE Hou et al. ( 2025 ) applies an iterative pruning strategy in RL training with an increasingly stringent token limit to reduce the reasoning length of long CoT LLMs. O1-Pruner Luo et al. ( 2025 ) , Kimi Team et al. ( 2025 ) , and ConciseR Song and Zheng ( 2025 ) incorporate length-based penalties into the reward function for RL training, incentivizing the model to produce more concise reasoning while maintaining accuracy. LC-R1 Cheng et al. ( 2025 ) incorporates a length reward into Group Relative Policy Optimization (GRPO), and a compress model to remove the invalid portion of the thinking process in training. 3 Methodology Figure 3: Example resoning trace of original Qwen3-8B and Draft thinking on a MATH500 question. 3.1 Draft-Thinking Formulation Let M M denote a reasoning large reasoning model (LRM) equipped with long chain-of-thought (CoT) capability. For any question 𝒒 \boldsymbol{q} , under a “step-by-step” reasoning prompt 𝒑 step \boldsymbol{p}_{\text{step}} , the model generates a sampled output 𝒐 ∼ M ​ ( 𝒑 step , 𝒒 ) \boldsymbol{o}\sim M(\boldsymbol{p}_{\text{step}},\boldsymbol{q}) , where 𝒐 = ( 𝒓 step ⊕ 𝒂 ) \boldsymbol{o}=(\boldsymbol{r}_{\text{step}}\oplus\boldsymbol{a}) consists of a reasoning trajectory 𝒓 step \boldsymbol{r}_{\text{step}} and a final answer 𝒂 \boldsymbol{a} . The objective of Draft-Thinking is to learn, within the same model, a draft-style reasoning behavior constrained by key reasoning, such that under a draft prompt 𝒑 draft \boldsymbol{p}_{\text{draft}} , the model generates 𝒐 ∼ M ​ ( 𝒑 draft , 𝒒 ) \boldsymbol{o}\sim M(\boldsymbol{p}_{\text{draft}},\boldsymbol{q}) , where 𝒐 = ( 𝒓 draft ⊕ 𝒂 ) \boldsymbol{o}=(\boldsymbol{r}_{\text{draft}}\oplus\boldsymbol{a}) . Here, 𝒓 draft \boldsymbol{r}_{\text{draft}} retains only the information that is necessary and decisive for the justifiability and correctness of 𝒂 \boldsymbol{a} . This formulation reduces the reasoning budget to improve efficiency and lays the foundation for subsequently elevating reasoning depth to a schedulable behavior. The overview of the proposed model is illustrated in Figure 7 . 3.2 Cultivating Draft Reasoning Capability Although prompt-based draft reasoning can reduce reasoning redundancy under specific settings, its effectiveness is highly sensitive to model scale and prompting conditions, making it difficult to serve as a transferable mechanism for injecting reasoning capability Xu et al. ( 2025 ) . To develop draft reasoning capability for model M M , we distill this ability from the larger model DeepSeek-V3-0324 (685B), using it to generate high-quality draft reasoning data for SFT. We start with LIMO Ye et al. ( 2025 ) , a curated mathematical dataset D = { ( 𝒒 i , 𝒂 i ) } i = 1 N D=\{(\boldsymbol{q}_{i},\boldsymbol{a}_{i})\}_{i=1}^{N} with N = 817 N=817 , selected from tens of millions of problems for its diverse difficulty, generality, and knowledge coverage. Using a carefully designed mathematical reasoning prompt method Chunked Symbolism Aytes et al. ( 2025 ) , we generate draft reasoning 𝒓 draft \boldsymbol{r}_{\text{draft}} and answers 𝒂 ^ \hat{\boldsymbol{a}} for D D . Filtering samples where 𝒂 ^ ≠ 𝒂 \hat{\boldsymbol{a}}\neq\boldsymbol{a} yields our SFT dataset D sft = { ( 𝒒 i , 𝒓 draft , 𝒂 i ) } i = 1 342 D_{\text{sft}}=\{(\boldsymbol{q}_{i},\boldsymbol{r}_{\text{draft}},\boldsymbol{a}_{i})\}_{i=1}^{342} . Subsequently, the distillation objective is to fine-tune model M M with dataset D sft D_{\text{sft}} to enable it to develop concise draft reasoning capability by minimizing the cross-entropy loss: ℒ = − ∑ i = 1 l log ​ P 𝜽 M ​ ( 𝒚 i | 𝒑 draft , 𝒒 , 𝒚 < i ) \mathcal{L}=-\sum_{i=1}^{l}\text{log}P_{\boldsymbol{\theta}_{M}}(\boldsymbol{y}_{i}|\boldsymbol{p}_{\text{draft}},\boldsymbol{q},\boldsymbol{y}_{D-RL3k draft 44.49 1133 3.92 43.96 6100 0.72 22.33 5002 0.45 57.78 3785 1.52 D-SFT->D-RL3k->D-RL6k draft 46.32 960 4.82 45.42 6527 0.69 22.33 6476 0.35 61.19 3433 1.78 step 53.68 5180 1.03 64.79 14112 0.46 30.83 13980 0.22 66.37 9519 0.69 adaptive 48.53 2979 1.62 56.67 12155 0.46 31.83 10341 0.31 69.19 6849 1.01 D-SFT->D-RL6k draft 49.26 1677 2.93 48.75 7426 0.65 26.33 6949 0.37 63.41 4340 1.46 step 53.68 5076 1.05 63.54 13760 0.46 31.33 13957 0.22 66.37 9170 0.72 adaptive 50.00 3209 1.55 61.88 12115 0.51 30.67 10741 0.28 67.85 7188 0.94 ThinkPrune(6k) step 53.68 3822 1.40 53.96 11720 0.46 28.67 11043 0.26 65.48 6902 0.94 ThinkPrune(6k->3k) step 54.04 2725 1.98 48.13 9781 0.49 28.67 9333 0.31 65.63 5809 1.13 REO-RL(Q-Spec)* step 46.80 3952 1.18 64.40 12645 0.51 - - - - - - R1-distill+FEDH* step - - - 46.70 14730 0.32 - - - 63.50 12353 0.51 R1-distill+Length-Penalty* step - - - 54.70 12446 0.44 - - - 68.40 7383 0.93 R1-distill+DR.SAF* step - - - 57.90 10692 0.54 - - - 71.30 5766 1.24 Table 1: Performance comparison across mathematical benchmarks based on Qwen3-8B. The best and second-best results per metric are shown in bold and underline , respectively. "*" indicates results from corresponding studies. 3.4 Inference After Progressive Curriculum Learning, model M M learns draft reasoning capability. At inference, the model can perform draft reasoning with prompt 𝒑 draft \boldsymbol{p}_{\text{draft}} or its original long CoT reasoning with 𝒑 step \boldsymbol{p}_{\text{step}} . Draft mode maximizes token efficiency while long CoT mode maximizes accuracy. We also design a hybrid instance adaptive prompt 𝒑 adaptive \boldsymbol{p}_{\text{adaptive}} that combines both modes, enabling the model to balance token efficiency with high accuracy by leveraging the strengths of each approach. Specifically, for each question 𝒒 \boldsymbol{q} , the model assesses difficulty and adaptively chooses between draft reasoning mode for simpler questions and long CoT mode for more challenging questions. The prompt designs are presented in Table 5 . 4 Experiments 4.1 Experiment Setup Method Base Model MATH500 GPQA-D Acc Tokens EFF Acc Tokens EFF Original Qwen3-8B 93.00 5668 1.64 55.05 11554 0.47 D-SFT (draft) Qwen3-8B 69.80 1732 4.02 41.92 1948 2.15 D-SFT->D-RL3k (draft) Qwen3-8B 89.40 1032 8.65 48.99 4504 1.08 D-SFT->D-RL3k->D-RL6k (draft) Qwen3-8B 90.60 986 9.18 54.04 2689 2.01 D-SFT->D-RL3k->D-RL6k (step) Qwen3-8B 93.60 4271 2.19 63.13 6620 0.95 D-SFT->D-RL3k->D-RL6k (adaptive) Qwen3-8B 93.20 2755 3.38 52.02 4800 1.08 D-SFT->D-RL6k (draft) Qwen3-8B 93.00 1453 6.39 57.07 3742 1.52 D-SFT->D-RL6k (step) Qwen3-8B 93.60 4539 2.06 57.58 7343 0.78 D-SFT->D-RL6k (adaptive) Qwen3-8B 94.20 2764 3.40 56.06 5219 1.07 ThinkPrune(6k) Qwen3-8B 93.60 3074 3.04 60.61 6322 0.96 ThinkPrune(6k->3k) Qwen3-8B 94.40 2615 3.61 60.10 5166 1.16 MUR* Qwen3-8B 93.80 5328 1.76 57.58 6147 0.93 DR.SAF* R1-Distill-Qwen3-8B 93.30 2168 4.30 - - - CTS(best EFF)* Qwen2.5-14B-Instruct 75.60 2036 3.71 46.50 2906 1.60 SimPO(FCS+Reflection)* QwQ-32B-Preview 92.80 1330 6.97 59.10 2085 2.83 s1-mix-32B* Qwen2.5-32B-Instruct 94.60 8648 1.09 61.10 21995 0.28 TOPS-Iter-DPO* Qwen2.5-32B-Instruct 91.60 1731 5.28 - - - ThinkPrune(RL)* QwQ-32B 93.80 2162 4.34 - - - O1-Pruner(RL)* QwQ-32B-Preview 91.00 1385 6.57 - - - Table 2: Performance comparison of our method with approaches based on the same backbone (Qwen3-8B) and larger backbones (14B-32B). The best and second-best results per metric are shown in bold and underline , respectively. "*" indicates results from corresponding studies. Backbone models. In our experiments, we use the long CoT reasoning models Qwen3-8B and Qwen3-4B Yang et al. ( 2025a ) as base models. Training datasets. Our training data includes three parts: D sft = { ( 𝒒 i , 𝒓 draft , 𝒂 i ) } i = 1 342 D_{\text{sft}}=\{(\boldsymbol{q}_{i},\boldsymbol{r}_{\text{draft}},\boldsymbol{a}_{i})\}_{i=1}^{342} for SFT training, and D rl = { ( 𝒒 i , 𝒂 i ) } i = 1 475 D_{\text{rl}}=\{(\boldsymbol{q}_{i},\boldsymbol{a}_{i})\}_{i=1}^{475} along with AIME2024 (30 samples) for RL training. Both D sft D_{\text{sft}} and D rl D_{\text{rl}} are constructed from the LIMO Ye et al. ( 2025 ) dataset as described in Section 3.2 and 3.3 , totaling 847 training samples. Evaluation configuration. We follow LIMO’s comprehensive evaluation framework to evaluate the effectiveness of our method. Our evaluation datasets include in-domain datasets (MATH500 Hendrycks et al. ( 2021 ) , AIME2025) and out-of-distribution datasets (LiveMathBench (version 202505) Liu et al. ( 2024 ) , OlympiadBench He et al. ( 2024 ) , MinervaMath Lewkowycz et al. ( 2022 ) , GPQA Rein et al. ( 2024 ) ). We evaluate model performance on all benchmarks using the pass@1 metric as accuracy (ACC) in a zero-shot setting. Additionally, we report the average response token length (LEN) and token efficiency (EFF). Token efficiency is defined as the ratio of accuracy to length (EFF = ACC / LEN × \times 100), serving as an indicator of the trade-off between correctness and reasoning efficiency. The details of the evaluation configuration are shown in Appendix A.2 . Comparisons. For comprehensive comparison, we evaluate our method against approaches on the same base model and larger models, encompassing three categories: (1) Online RL methods , (2) Offline methods and (3) Training-free methods . The details of baselines are shown in Appendix A.3 . 4.2 Main Experimental Results Draft-Thinking achieves significantly superior token efficiency (EFF) compared to all other methods . As shown in Tables 1 - 2 , Draft-Thinking attains the highest EFF across all mathematical reasoning benchmarks. For instance, on MATH500, Draft-Thinking achieves an EFF of 9.18, substantially outperforming other RL methods on the same base model (e.g., ThinkPrune and DR.SAF), and even surpassing RL-based (O1-Pruner and ThinkPrune) and offline (SimPO(FCS+Reflection)) methods on larger 32B models. Draft-Thinking dramatically improves EFF with minimal accuracy loss. On MATH500, accuracy decreases by only 2.58% compared to the original Qwen3-8B baseline, while token count is reduced by 82.6%, yielding a 5.6× EFF improvement. Draft-Thinking simultaneously enhances the model’s long-CoT accuracy while substantially boosting its token efficiency . After three stages of Draft-Thinking training, the model exhibits improved long CoT reasoning (step prompt) performance across all benchmarks while demonstrating substantial reductions in token length relative to the original baseline. Notably, on the most challenging benchmark AIME2025 (Table 1 ), Draft-Thinking achieves a long CoT accuracy of 64.79, marginally exceeding the baseline’s 64.38, while reducing token consumption by 3,922 tokens (21.7%). On MinervaMath, accuracy of long CoT reasoning improves by 1.47 points with a concurrent 32% decrease in token usage. The adaptive prompt approach leverages the advantages of both draft and original long CoT modes, achieving a balanced reasoning performance . After three stages of Draft-Thinking training, the model’s draft reasoning attains optimal token efficiency while its long CoT reasoning attains the best accuracy. As shown in Tables 1 - 2 , the adaptive prompt method achieves higher EFF than long CoT reasoning and higher accuracy than draft reasoning across all mathematical reasoning benchmarks. Notably, the adaptive prompt method achieves both higher accuracy and higher EFF than long CoT reasoning on OlympiadBench and MATH500. Draft-Thinking exhibits robust generalization capabilities across out-of-domain benchmarks . Beyond strong performance on out-of-distribution mathematical reasoning benchmarks, it also demonstrates effectiveness on the non-mathematical reasoning task GPQA-D. As illustrated in Table 2 , Draft-Thinking’s draft reasoning mode incurs merely a 1.83% accuracy degradation (55.05→54.04) while achieving a 76.7% token reduction (11,554→2,689). Moreover, its long CoT reasoning mode delivers a substantial 14.68% accuracy improvement (55.05→63.13) along with a 42.7% decrease in token usage (11,554→6,620). Figure 3 presents a detailed case comparison between Qwen3-8B and Draft-Thinking on a MATH500 sample. Method Prompt Minerva AIME2025 LiveMathBench MATH500 Acc Tokens EFF Acc Tokens EFF Acc Tokens EFF Acc Tokens EFF Qwen3-4B Original step 51.47 7253 0.71 63.12 17703 0.36 29 16813 0.17 92.2 5505 1.67 D-SFT->D-RL3k->D-RL6k draft 46.69 1421 3.28 46.04 7097 0.65 22.67 6051 0.38 92 1269 7.24 step 48.9 4815 1.01 60.21 12885 0.46 26.67 12863 0.21 92.8 4130 2.24 adaptive 48.9 3620 1.35 53.33 10048 0.53 24.17 8353 0.29 92.8 2292 4.04 D-RL3k->D-RL6k draft 49.26 1669 2.95 44.79 7720 0.58 21.17 6242 0.34 92.4 1531 6.03 step 51.84 3716 1.39 51.67 12186 0.42 27.83 11626 0.24 92.8 3309 2.80 adaptive 51.84 2206 2.34 49.17 10304 0.47 23.17 8751 0.26 92 2342 3.92 D-RL6k->D-RL3k draft 50 1323 3.77 45 7787 0.57 19.17 6292 0.30 92.2 1525 6.04 step 49.63 3936 1.26 54.37 11731 0.46 26.5 11626 0.23 93.2 2952 3.15 adaptive 50 2392 2.08 48.13 9903 0.48 26.33 8358 0.31 91.4 2262 4.04 RL6k->RL3k step 51.47 2785 1.84 47.71 10951 0.43 21.83 9528 0.23 92 2598 3.54 Table 3: Comparison of different training strategies on Qwen3-4B. 4.3 Ablation Study and Analysis 4.3.1 Progressive Curriculum Learning Progressive curriculum learning achieved the best token efficiency. We present the draft reasoning performance of three training stages in Tables 1 - 2 , and compare it with a two-stage variant (D-SFT→D-RL6k). Compared to the original model, D-SFT (after Draft SFT) generates significantly fewer tokens but with decreased accuracy, as the draft CoT constructed by the larger DeepSeek-V3 (685B) model is not optimal for the 8B model. After the first RL stage (D-SFT→D-RL3k), draft reasoning accuracy improves substantially. For instance, on the challenging AIME2025 dataset, accuracy increases from 9.17 to 43.96, with EFF rising from 0.23 to 0.72. Following the second RL stage with a longer maximum output length and the more challenging AIME2024 training set, the model not only achieves further accuracy improvements across multiple datasets but also reduces token counts. For example, on GPQA-D, EFF nearly doubles. Additionally, the two-stage variant D-SFT→D-RL6k, which omits RL training at 3k maximum length, achieves comparable accuracy to the three-stage Draft-Thinking method across all datasets but generates substantially more tokens, resulting in significantly lower EFF than the three-stage approach. This demonstrates that iterative reinforcement learning more efficiently cultivates draft reasoning capability. Figure 8 and 9 present the accuracy, token count, and token efficiency trends across the three Draft-Thinking training stages for three reasoning modes using Qwen3-8B and Qwen3-4B as the base model. We observe three key patterns: First, as shown in the top row of Figure 8 , Draft-Thinking training maintains the original long-CoT performance while substantially boosting its token efficiency. Second, two-stage RL training yields universal accuracy gains in draft reasoning, with token counts decreasing across most benchmarks except for AIME2025 and LiveMathBench. This suggests that harder tasks necessitate training with larger maximum lengths. Third, EFF gains in draft reasoning drive improvements in both long-CoT and adaptive reasoning. 4.3.2 Training Pipeline Analysis To comprehensively evaluate the advantages and limitations of Draft-Thinking, we conduct comparative experiments on Qwen3-4B with several method variants. Specifically, we assess: (1) D-RL3k→D-RL6k , which excludes the Draft SFT stage and conducts two-stage RL training directly with the draft prompt; (2) ThinkPrune (RL6k→RL3k) , a multi-stage RL approach employing iterative CoT length pruning; and (3) D-RL6k→D-RL3k , an enhanced ThinkPrune variant trained under the draft prompt. To ensure fair comparison, all methods utilize identical training datasets at the 3k and 6k length configurations: D rl D_{\text{rl}} and AIME2024, respectively. Table 3 demonstrates that the complete Draft-Thinking method attains the highest EFF on three datasets and second-highest on another, exhibiting substantially superior overall performance relative to alternative approaches. The variant D-RL3k→D-RL6k, which excludes Draft SFT, produces inferior draft reasoning EFF on four datasets and shows a significant 14% (60.21 ->51.67) accuracy degradation in long CoT performance on AIME2025 compared to the complete method. This demonstrates that Draft SFT elevates the performance ceiling of draft reasoning EFF and plays a crucial role in enabling mode discrimination, thereby preserving long CoT reasoning quality . 4.4 Reasoning Behavior Analysis 4.4.1 Comparative Reasoning Behavior Figure 4: Reasoning behavior comparison between original Qwen3-8B and Draft thinking on MATH500. Each bar represents the cumulative number of reasoning steps within a phase category. To gain deeper insight into Draft-Thinking’s efficient reasoning mechanism, we analyze the reasoning behavior differences between the original model and Draft-Thinking’s three reasoning modes. Specifically, for each model-generated response, we prompt DeepSeek-V3-0324 to segment it into distinct phases across 10 predefined categories Hou et al. ( 2025 ) , such as "Understanding the Problem" and "Computing or Simplifying Expressions". We then quantify the number of reasoning steps within each phase by counting double newlines (" \n\n ") as step delimiters. Figure 4 reveals that draft reasoning mode significantly reduces the total number of reasoning steps relative to the original model. It concentrates on core problem-solving phases, such as "Computing or Simplifying Expressions," while substantially reducing redundant phases like "Exploring Alternative Approaches". Long CoT reasoning decreases step counts across most phases, except "Computing or Simplifying Expressions" and "Exploring Alternative Approaches". Compared to draft reasoning, adaptive reasoning primarily increases the "Reassess and Verify Local Steps" phase. Merics Original Draft Step Adaptive Avg Steps 111 22 88 50 Avg Tokens per Step 51 45 48 55 Table 4: Average number of reasoning steps and average tokens per step on MATH500. We use double newlines (" \n\n ") as a step delimiter. As shown in Table 4 , Draft-Thinking achieves token efficiency primarily through a substantial reduction in the number of reasoning steps. Moreover, in draft reasoning mode, each individual step is more concise compared to the original model, as reflected in the lower average tokens per step. 4.4.2 Difficulty-Level Analysis Figure 5: Comparison of average accuracy and response length across different difficulty levels on MATH500. Figure 6: Average lengths of correct vs. wrong responses on MATH500. As illustrated in Figure 5 , compared to the original model, draft reasoning only exhibits a decline in accuracy on high-difficulty problems (Levels 4 and 5), whereas adaptive reasoning and long CoT reasoning demonstrate comparable performance. Notably, the average response length decreases most significantly on Level 1-3 problems; for instance, it is reduced to one-tenth on Levels 1-2 problems. Figure 6 highlights a substantial disparity in the average lengths of correct and wrong responses, with wrong responses being significantly longer than correct ones. Consequently, improving the accuracy of Draft reasoning on high-difficulty tasks necessitates training with a larger maximum response length. 5 Conclusion In this work, we propose Draft-Thinking , which internalizes the selection of high-value steps as an endogenous capability and reasoning pattern of LRMs, significantly outperforming existing methods in reasoning efficiency. Draft-Thinking maintains the original long-CoT capability while substantially boosting its token efficiency. Moreover, Draft-Thinking introduces adaptive prompting, which elevates reasoning depth to a flexible, model-selectable behavior. Limitations Draft-Thinking does not compromise the original long-CoT capability and even improves token efficiency across all benchmarks. However, on the most difficult AIME 2025 dataset, the accuracy gap between draft and long-CoT modes is larger than on other sets. We suspect that complex problems require a larger maximum sequence length during draft reasoning training. For instance, on the simpler Math500, draft reasoning already performs nearly as well as long-CoT. Limited by computational resources, we did not further explore the potential of draft reasoning with even longer maximum sequence length. References D. Arora and A. Zanette (2025) Training language models to reason efficiently . arXiv preprint arXiv:2502.04463 . Cited by: §A.3 . S. A. Aytes, J. Baek, and S. J. Hwang (2025) Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching . arXiv . Note: arXiv:2503.05179 [cs] version: 1 External Links: Link , Document Cited by: §A.5 , Figure 11 , §1 , §2 , §3.2 . M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. (2021) Evaluating large language models trained on code . arXiv preprint arXiv:2107.03374 . Cited by: §A.2 . Q. Chen, D. Peng, J. Liu, H. Su, J. Guan, L. Qin, and W. Che (2025a) Aware First, Think Less: Dynamic Boundary Self-Awareness Drives Extreme Reasoning Efficiency in Large Language Models . arXiv . Note: arXiv:2508.11582 [cs] External Links: Link , Document Cited by: §A.3 . X. Chen, J. Xu, T. Liang, Z. He, J. Pang, D. Yu, L. Song, Q. Liu, M. Zhou, Z. Zhang, R. Wang, Z. Tu, H. Mi, and D. Yu (2025b) Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs . arXiv . Note: arXiv:2412.21187 [cs] External Links: Link , Document Cited by: §A.3 , §1 , §2 . Z. Cheng, D. Chen, M. Fu, and T. Zhou (2025) Optimizing Length Compression in Large Reasoning Models . arXiv . Note: arXiv:2506.14755 [cs] External Links: Link , Document Cited by: §2 . G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen, et al. (2025) Gemini 2.5: pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities . arXiv preprint arXiv:2507.06261 . Cited by: §1 . A. Cuadron, D. Li, W. Ma, X. Wang, Y. Wang, S. Zhuang, S. Liu, L. G. Schroeder, T. Xia, H. Mao, N. Thumiger, A. Desai, I. Stoica, A. Klimovic, G. Neubig, and J. E. Gonzalez (2025) The Danger of Overthinking: Examining the Reasoning-Action Dilemma in Agentic Tasks . arXiv . Note: arXiv:2502.08235 [cs] External Links: Link , Document Cited by: §1 , §2 . M. Fatemi, B. Rafiee, M. Tang, and K. Talamadupula (2025) Concise Reasoning via Reinforcement Learning . arXiv . Note: arXiv:2504.05185 [cs] External Links: Link , Document Cited by: §3.3 . J. Gao, S. Yan, Q. Tan, L. Yang, S. Xu, W. Fu, Z. Mei, K. Lyu, and Y. Wu (2025) How Far Are We from Optimal Reasoning Efficiency? . arXiv . Note: arXiv:2506.07104 [cs] External Links: Link , Document Cited by: §A.3 . D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. (2025) Deepseek-r1: incentivizing reasoning capability in llms via reinforcement learning . arXiv preprint arXiv:2501.12948 . Cited by: §A.2 , §1 . T. Han, Z. Wang, C. Fang, S. Zhao, S. Ma, and Z. Chen (2025) Token-Budget-Aware LLM Reasoning . arXiv . Note: arXiv:2412.18547 [cs] External Links: Link , Document Cited by: §2 . C. He, R. Luo, Y. Bai, S. Hu, Z. L. Thai, J. Shen, J. Hu, X. Han, Y. Huang, Y. Zhang, et al. (2024) Olympiadbench: a challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems . arXiv preprint arXiv:2402.14008 . Cited by: §4.1 . D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt (2021) Measuring mathematical problem solving with the math dataset . arXiv preprint arXiv:2103.03874 . Cited by: §4.1 . B. Hou, Y. Zhang, J. Ji, Y. Liu, K. Qian, J. Andreas, and S. Chang (2025) ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning . arXiv . Note: arXiv:2504.01296 [cs] External Links: Link , Document Cited by: §A.3 , §A.6 , Figure 12 , §1 , §2 , §4.4.1 . Y. Kang, X. Sun, L. Chen, and W. Zou (2024) C3oT: Generating Shorter Chain-of-Thought without Compromising Effectiveness . arXiv . Note: arXiv:2412.11664 [cs] External Links: Link , Document Cited by: §2 . A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, et al. (2022) Solving quantitative reasoning problems with language models . Advances in neural information processing systems 35 , pp. 3843–3857 . Cited by: §4.1 . Z. Ling, D. Chen, H. Zhang, Y. Jiao, X. Guo, and Y. Cheng (2025) Fast on the easy, deep on the hard: efficient reasoning via powered length penalty . arXiv preprint arXiv:2506.10446 . Cited by: §A.3 . J. Liu, H. Liu, L. Xiao, Z. Wang, K. Liu, S. Gao, W. Zhang, S. Zhang, and K. Chen (2024) Are your llms capable of stable reasoning? . arXiv preprint arXiv:2412.13147 . Cited by: §4.1 . H. Luo, L. Shen, H. He, Y. Wang, S. Liu, W. Li, N. Tan, X. Cao, and D. Tao (2025) O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning . arXiv . Note: arXiv:2501.12570 [cs] External Links: Link , Document Cited by: §A.3 , §1 , §2 . G. A. Miller (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. . Psychological review 63 ( 2 ), pp. 81 . Cited by: §A.5 . OpenAI (2025) GPT-5 system card . Technical report OpenAI . Note: Accessed: 2025-09 External Links: Link Cited by: §1 . D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman (2024) Gpqa: a graduate-level google-proof q&a benchmark . In First Conference on Language Modeling , Cited by: §4.1 . Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo (2024) DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models . arXiv . Note: arXiv:2402.03300 [cs] External Links: Link , Document Cited by: §3.3 . G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu (2025) Hybridflow: a flexible and efficient rlhf framework . In Proceedings of the Twentieth European Conference on Computer Systems , pp. 1279–1297 . Cited by: §A.4 . M. Song and M. Zheng (2025) Walk Before You Run! Concise LLM Reasoning via Reinforcement Learning . arXiv . Note: arXiv:2505.21178 [cs] External Links: Link , Document Cited by: §1 , §2 . K. Team, A. Du, B. Gao, B. Xing, C. Jiang, C. Chen, C. Li, C. Xiao, C. Du, C. Liao, C. Tang, C. Wang, D. Zhang, E. Yuan, E. Lu, F. Tang, F. Sung, G. Wei, G. Lai, H. Guo, H. Zhu, H. Ding, H. Hu, H. Yang, H. Zhang, H. Yao, H. Zhao, H. Lu, H. Li, H. Yu, H. Gao, H. Zheng, H. Yuan, J. Chen, J. Guo, J. Su, J. Wang, J. Zhao, J. Zhang, J. Liu, J. Yan, J. Wu, L. Shi, L. Ye, L. Yu, M. Dong, N. Zhang, N. Ma, Q. Pan, Q. Gong, S. Liu, S. Ma, S. Wei, S. Cao, S. Huang, T. Jiang, W. Gao, W. Xiong, W. He, W. Huang, W. Wu, W. He, X. Wei, X. Jia, X. Wu, X. Xu, X. Zu, X. Zhou, X. Pan, Y. Charles, Y. Li, Y. Hu, Y. Liu, Y. Chen, Y. Wang, Y. Liu, Y. Qin, Y. Liu, Y. Yang, Y. Bao, Y. Du, Y. Wu, Y. Wang, Z. Zhou, Z. Wang, Z. Li, Z. Zhu, Z. Zhang, Z. Wang, Z. Yang, Z. Huang, Z. Huang, Z. Xu, and Z. Yang (2025) Kimi k1.5: Scaling Reinforcement Learning with LLMs . arXiv . Note: arXiv:2501.12599 [cs] External Links: Link , Document Cited by: §2 . H. Xia, Y. Li, C. T. Leong, W. Wang, and W. Li (2025) TokenSkip: Controllable Chain-of-Thought Compression in LLMs . arXiv . Note: arXiv:2502.12067 [cs] External Links: Link , Document Cited by: §1 , §2 . S. Xu, W. Xie, L. Zhao, and P. He (2025) Chain of Draft: Thinking Faster by Writing Less . arXiv . Note: arXiv:2502.18600 [cs] External Links: Link , Document Cited by: §1 , §2 , §3.2 . H. Yan, F. Xu, R. Xu, Y. Li, J. Zhang, H. Luo, X. Wu, L. A. Tuan, H. Zhao, Q. Lin, and J. Liu (2025) MUR: Momentum Uncertainty guided Reasoning for Large Language Models . arXiv . Note: arXiv:2507.14958 [cs] External Links: Link , Document Cited by: §A.3 . A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Gao, C. Huang, C. Lv, C. Zheng, D. Liu, F. Zhou, F. Huang, F. Hu, H. Ge, H. Wei, H. Lin, J. Tang, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Zhou, J. Lin, K. Dang, K. Bao, K. Yang, L. Yu, L. Deng, M. Li, M. Xue, M. Li, P. Zhang, P. Wang, Q. Zhu, R. Men, R. Gao, S. Liu, S. Luo, T. Li, T. Tang, W. Yin, X. Ren, X. Wang, X. Zhang, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Zhang, Y. Wan, Y. Liu, Z. Wang, Z. Cui, Z. Zhang, Z. Zhou, and Z. Qiu (2025a) Qwen3 technical report . External Links: 2505.09388 , Link Cited by: §1 , §4.1 . W. Yang, S. Ma, Y. Lin, and F. Wei (2025b) Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning . arXiv . Note: arXiv:2502.18080 [cs] External Links: Link , Document Cited by: §A.3 , §2 . Y. Ye, Z. Huang, Y. Xiao, E. Chern, S. Xia, and P. Liu (2025) LIMO: Less is More for Reasoning . arXiv . Note: arXiv:2502.03387 [cs] External Links: Link , Document Cited by: §3.2 , §4.1 . B. Yu, H. Yuan, H. Li, X. Xu, Y. Wei, B. Wang, W. Qi, and K. Chen (2025) Long-Short Chain-of-Thought Mixture Supervised Fine-Tuning Eliciting Efficient Reasoning in Large Language Models . arXiv . Note: arXiv:2505.03469 [cs] External Links: Link , Document Cited by: §A.3 . H. Yuan, B. Yu, H. Li, S. Yang, C. D. Wang, Z. Yu, X. Xu, W. Qi, and K. Chen (2025) Not All Tokens Are What You Need In Thinking . arXiv ( en ). Note: Version Number: 1 External Links: Link , Document Cited by: §A.3 , §1 , §2 . Appendix A Experimental Setting Details Figure 7: Overview of Progressive Curriculum Learning. Stage 1 : Distill draft reasoning capability from a larger teacher model through SFT. Stages 2-3 : Enhance draft reasoning through iterative RL with increasing maximum response lengths and progressively challenging datasets. Inference : Step prompt enables long CoT reasoning, draft prompt enables draft reasoning, and adaptive prompt combines both modes. A.1 Evaluation Dataset Detail • MinervaMath : An undergraduate-level dataset containing 272 problems in physics, biology, chemistry, economics, and other sciences that require quantitative reasoning. • OlympiadBench : A subset of the original OlympiadBench dataset containing 675 problems from Olympiad-level mathematics and physics competitions, including the Chinese college entrance examination. • MATH500 : A challenging benchmark of 500 high-school competition-level problems spanning seven subjects, including Algebra, Geometry, Number Theory, and Precalculus. Each problem is presented in natural language with LaTeX-formatted notation, providing a robust measure of mathematical reasoning and generalization across diverse topics. • AIME2025 : A dataset comprising 30 problems from the 2025 American Invitational Mathematics Examination (AIME), a prestigious high-school mathematics competition for top-performing students. Each problem requires deep mathematical insight, multi-step reasoning, and precise problem-solving skills. • LiveMathBench : LiveMathBench is a mathematical dataset specifically designed to include challenging problems from the latest mathematical competitions, thereby avoiding data contamination issues prevalent in existing LLMs and public math benchmarks. We use the 202505 version of LiveMathBench, which contains 100 mathematical questions from various countries, including non-English problems. • GPQA-Diamond : A challenging dataset consisting of 198 graduate-level multiple-choice questions written by domain experts across biology, physics, and chemistry. Prompt Method Content Long CoT Please reason step by step, and put your final answer within \boxed{}. Draft Let’s think step by step, but only keep a minimum draft for each thinking step, with as few words as possible, and output the final answer within \boxed{}. Instance adaptive First, you should decide the thinking mode based on the question’s difficulty. The first mode is the normal way, where you can think in detail. The second mode is to keep only a minimal draft with as few words as possible. Then reason step by step, and output the final answer within \boxed{}. Table 5: Detailed designs of three prompts: 𝒑 step \boldsymbol{p}_{\text{step}} for long CoT reasoning, 𝒑 draft \boldsymbol{p}_{\text{draft}} for draft reasoning, and 𝒑 adaptive \boldsymbol{p}_{\text{adaptive}} for instance-adaptive reasoning. Figure 8: Accuracy, token count, and token efficiency trends for three reasoning modes (draft, long CoT, and adaptive) across the three training stages of Draft-Thinking on Qwen3-8B. Figure 9: Accuracy, token count, and token efficiency trends for three reasoning modes (draft, long CoT, and adaptive) across the three training stages of Draft Thinking on Qwen3-4B. A.2 Evaluation Configuration Detail For larger benchmarks (MATH500, Minerva Math, OlympiadBench, GPQA), we employ greedy decoding with a single sample. For the smaller AIME2025 benchmark (30 problems) and LiveMathBench (100 problems), we generate 16 and 6 samples, respectively, with a temperature of 0.6 and top-p value of 0.95, then compute the unbiased pass@1 metric Chen et al. ( 2021 ) . We set the maximum generation length (including both reasoning and answer tokens) to 32,768 tokens for all models, which significantly exceeds the maximum token length (6,000) used during training. We adopt a mathematical evaluator based on Qwen-2.5-Math Guo et al. ( 2025 ) , which provides robust answer extraction and advanced expression comparison for complex evaluation scenarios. A.3 Comparisons Detail Online RL methods : O1-Pruner Luo et al. ( 2025 ) , FEDH Ling et al. ( 2025 ) , and Length-Penalty Arora and Zanette ( 2025 ) employ length-based rewards. REO-RL Gao et al. ( 2025 ) optimizes token budget selection. DR.SAF Chen et al. ( 2025a ) introduces a dynamic reasoning-boundary self-awareness framework. ThinkPrune Hou et al. ( 2025 ) uses iterative length pruning to preserve model performance. Offline methods : CTS Yuan et al. ( 2025 ) compresses CoT reasoning. TOPS-Iter-DPO Yang et al. ( 2025b ) performs Direct Preference Optimization using preference pairs containing the shortest correct responses. SimPO (FCS+Reflection) Chen et al. ( 2025b ) retains only the first and second correct solutions from QWQ-32B-Preview responses and retrains to mitigate overthinking. s1-mix-32B Yu et al. ( 2025 ) constructs a mixed dataset of long and short CoT reasoning for efficient SFT. Training-free methods : MUR Yan et al. ( 2025 ) dynamically allocates thinking budgets to critical reasoning steps to guide efficient LLM reasoning. For our Draft-Thinking approach, we report the performance of its three training stages, with maximum lengths set to 3,000 and 6,000 tokens for the two RL stages, respectively. We also evaluate a variant that performs SFT training on D sft D_{\text{sft}} followed by a single RL training stage on D rl D_{\text{rl}} at 6,000 tokens maximum length. For fair comparison with our method, we set ThinkPrune’s maximum length to 6,000 tokens (one-shot version) and 6,000→3,000 tokens (iterative pruning version). A.4 Implementation details. We utilize the Verl Sheng et al. ( 2025 ) framework for both supervised fine-tuning and reinforcement learning on 6 L20-40G GPUs. Table 6 shows the detailed training parameters for the Verl framework. A.5 Chunked Symbolism Aytes et al. ( 2025 ) Chunked symbolism is based on working memory chunking theory Miller ( 1956 ) , which condenses mathematical reasoning into dense symbolic representations containing more information with fewer tokens. The detailed prompt of Chunked Symbolism is shown in Figure 11 . A.6 Reasoning behavior analysis prompt Hou et al. ( 2025 ) Figure 12 shows the detail prompt for reasoning behavior analysis. Appendix B Additional Experiments Figure 10 shows the evolution of response length and validation accuracy during RL training stages on Qwen3-8B. Figure 10: Evolution of response length and validation accuracy during RL training. Notably, D-SFT->D-RL3k sustains concise response lengths while maintaining high accuracy from the beginning. The subsequent D-RL6k stage further enhances performance. Parameter D-RL3k D-RL6k datasets D rl D_{\text{rl}} AIME2024 advantage estimator grpo train batch size 96 30 max prompt length 240 500 max response length 3000 6000 actor optim lr 1e-6 use remove padding True ppo mini batch size 48 30 ppo micro batch size per gpu 2 1 use kl loss False entropy coeff 0 enable gradient checkpointing True model parallel size 2 actor rollout engine vllm rollout num per question 6 gpu memory utilization 0.6 0.5 use kl in reward False num nodes 1 gpus per node 6 (L20-48G) total epochs 15 60 Table 6: Main parameters of the Verl Training Framework for Qwen3-8B. Figure 11: Chunked Symbolism Prompt Aytes et al. ( 2025 ) . Figure 12: Reasoning behavior analysis prompt Hou et al. ( 2025 ) . BETA