Title: 2604.05164 ArXiv: 2604.05164 Not All Turns Are Equally Hard: Adaptive Thinking Budgets For Efficient Multi-Turn Reasoning Title: Content selection saved. Describe the issue below: Description: License: CC BY-NC-ND 4.0 arXiv:2604.05164v2 [cs.LG] 14 Apr 2026 Not All Turns Are Equally Hard: Adaptive Thinking Budgets For Efficient Multi-Turn Reasoning Neharika Jali, Anupam Nayak & Gauri Joshi Carnegie Mellon University {njali, anupamn, gaurij}@andrew.cmu.edu Abstract As LLM reasoning performance plateau, improving inference-time compute efficiency is crucial to mitigate overthinking and long thinking traces even for simple queries. Prior approaches including length regularization, adaptive routing, and difficulty-based budget allocation primarily focus on single-turn settings and fail to address the sequential dependencies inherent in multi-turn reasoning.In this work, we formulate multi-turn reasoning as a sequential compute allocation problem and model it as a multi-objective Markov Decision Process. We propose TAB: Turn-Adaptive Budgets, a budget allocation policy trained via Group Relative Policy Optimization (GRPO) that learns to maximize task accuracy while respecting global per-problem token constraints. Consequently, TAB takes as input the conversation history and learns to adaptively allocate smaller budgets to easier turns and save appropriate number of tokens for the crucial harder reasoning steps. Our experiments on mathematical reasoning benchmarks demonstrate that TAB achieves a superior accuracy-tokens tradeoff saving up to 35% tokens while maintaining accuracy over static and off-the-shelf LLM budget baselines. Further, for systems where a plan of all sub-questions is available apriori, we propose TAB All-SubQ, a budget allocation policy that budgets tokens based on the conversation history and all past and future sub-questions saving up to 40 % 40\% tokens over baselines. 1 Introduction Figure 1: On the left is a multi-turn reasoning trajectory with a sub-question of varying difficulty presented at every turn. Adaptive difficulty level based per-turn token budget allocations result in improved utilization of compute. On the right is a multi-turn system where at every turn t t at inference (colored blocks), the user presents a sub-question q t q_{t} , the budget allocation policy allots a token budget b t b_{t} based on the conversation history x 1 : t − 1 x_{1:t-1} and q t q_{t} to the solver which then generates a response y t y_{t} . The Budgeter is trained via GRPO with a reward function that is a combination of accuracy and number of tokens used. As recent gains in LLM reasoning increasingly come from test-time computation (Wu et al. , 2025 ; Muennighoff et al. , 2025 ; Snell et al. , 2024 ) , a central deployment question is no longer only how to scale inference, but how to spend that inference budget well. Efficient reasoning matters not only because it reduces cost, but also because it enables more effective test-time scaling under fixed resources. The same compute budget can be used more productively by reserving deeper reasoning for the steps that matter most, or by supporting more candidate solutions and more requests overall. This also matters at the systems level. Under real-world multi-user serving workloads, unnecessary reasoning increases latency, memory use, and long-context processing costs, which reduces throughput (Agrawal et al. , 2024 ; Sun et al. , 2024 ) . Prior systems work further shows that these serving bottlenecks become more pronounced for longer sequences and long-context inference (Kwon et al. , 2023 ; Hooper et al. , 2025 ) . Current approaches to efficient test-time reasoning largely follow a common philosophy. They first estimate the difficulty of a problem, then allocate an appropriate amount of compute, and finally solve the problem under that budget (Wang et al. , 2025b ; Shen et al. , 2025 ; Zhang et al. , 2025b ) . In this sense, most existing methods treat efficient reasoning as a one-shot allocation problem at the level of an entire instance. Multi-turn reasoning, often introduced via task or sub-question decomposition to improve performance, makes efficient compute allocation even more important. In single-turn settings, wasted compute is local and an unnecessarily long trace only slows and increases the cost of one response. In multi-turn settings, however, inefficiency compounds over time (Jeong and Ahn, 2025 ; Gao et al. , 2024 ) . Tokens generated early become part of later context, increasing memory, bandwidth, and long-context serving costs as the interaction proceeds (Agrawal et al. , 2024 ; Qin et al. , 2025 ) . This is especially problematic in decomposed pipelines, where some sub-questions are routine while others are decisive (Zeng et al. , 2025 ) ; uniform or myopic per-turn budgets can therefore waste compute on bookkeeping steps and under-invest in the turns that matter most. Yet, compared with single-turn efficient reasoning, multi-turn budget allocation remains relatively underexplored. In this work, we cast multi-turn reasoning as a distinct sequential compute allocation problem. This setting is challenging for several reasons. First, it is characterized by strong temporal dependency across turns. A budget decision at the current turn affects not only the immediate response quality, but also the remaining compute, the downstream reasoning trajectory, and the final task outcome, all without visibility into future sub-questions. Second, feedback is delayed. Since rewards are observed, or correctness is verified, only at the end of the trajectory, the problem involves difficult credit assignment across turn-level decisions. Finally, the objective is inherently multi-objective . A successful approach must jointly optimize task accuracy and token efficiency over entire reasoning trajectories. Together, these properties make multi-turn budget allocation fundamentally different from single-turn budget selection and render existing single-turn efficiency methods non-trivial to extend to this setting. Main Contributions. We address the problem of compute efficiency in multi-turn reasoning in the following manner. 1. We formulate multi-turn reasoning as a sequential compute allocation problem, formalized as a multi-objective Markov Decision Process (MDP). We propose TAB: Turn-Adaptive Budgets, a budget allocation policy trained via Group Relative Policy Optimization (GRPO) that learns to maximize task accuracy while respecting global per-problem token constraints. 2. We demonstrate that TAB achieves a superior accuracy-tokens tradeoff compared to static and off-the-shelf LLM-Judge baselines saving up to 35 % 35\% tokens while improving or maintaining accuracy on math reasoning benchmarks. 3. For systems where a plan of all sub-questions is available apriori, we propose a variant TAB All-SubQ, a budget allocation policy that allots tokens as a function of the conversation trajectory thus-far and all sub-questions. It saves up to 40 % 40\% tokens over baselines strengthening our claim of multi-turn reasoning efficiency being a planning problem rather than an individual difficulty-estimation problem. 2 Related Works Inference Time Budget Adaptation Methods. Inference-time interventions are increasingly used to improve reasoning efficiency by allocating computation more selectively. One common strategy is model routing, which directs easier queries to smaller or specialized LLMs while reserving larger models for more difficult instances (Ding et al. , 2024 ; Zhang et al. , 2025a ; Shao et al. , 2024 ; Ong et al. , 2025 ; Patel et al. , 2025 ) . Related work has also explored token-level routing, which enables finer-grained compute allocation within a single generation rather than only at the query level (Kapoor et al. , 2026 ) . These ideas have further been extended to agentic settings, where routing decisions must additionally account for downstream execution costs (Zhang et al. , 2026 ; Qian et al. , 2025 ) . Another line of work focuses on estimating question difficulty before assigning a reasoning budget (Wang et al. , 2025b ; Shen et al. , 2025 ; Huang et al. , 2025 ) , while other approaches dynamically switch between thinking and non-thinking modes based on estimated difficulty (Zhang et al. , 2025b ) . More recently, adaptive budget forcing methods have been proposed, using signals such as confidence or token entropy to decide whether additional reasoning is needed (Yuan et al. , 2025 ; Li et al. , 2026 ) . However, most of these methods are designed for single-turn interactions and do not naturally support planning or budget allocation across multi-turn settings, where reasoning costs must be managed adaptively over the course of an interaction. Further, methods for model routing in multi-turn conversations often incur significant infrastructure overheads of hosting multiple model variants, the prefill and/or decode latency associated with cross-model transitions and cost of extra model invocations. Inference Time Planning. Complementary to the budget adaptation methods are task Task decomposition (TD) methods that aim to improve performance by decomposing difficult problems into simpler subproblems that can be solved sequentially. Well before GRPO-style post-training became prevalent, prompting-based TD methods had already shown substantial benefits from explicitly structuring intermediate reasoning (Wei et al. , 2022 ; Radhakrishnan et al. , 2023 ; Wang et al. , 2023 ; Khot et al. , 2023 ) . More recent reasoning-oriented work has revisited this paradigm by using a stronger model to generate a decomposition or plan and a smaller model to execute the corresponding subtasks (Lin et al. , 2026 ; Xin et al. , 2025 ) . This strategy has also been widely adopted in agent-based frameworks (Qian et al. , 2025 ; Erdogan et al. , 2025 ) . More recently, hierarchical RL approaches have sought to endow models with subgoal decomposition capabilities through training (Wang et al. , 2025a ; Ge et al. , 2025 ; Ren et al. , 2025 ) . Nevertheless, the primary focus of most of these methods has been on improving task performance, rather than on enabling more efficient reasoning. Training Efficient Reasoners. Following the observation that GRPO-based post-training tends to increase the average length of model responses as training progresses, a growing body of work has sought to mitigate overthinking in LLMs and improve token-efficient reasoning, especially on easier problems. One prominent line of research modifies the training objective by incorporating length penalties or related mechanisms to encourage shorter reasoning traces (Arora and Zanette, 2025 ; Aggarwal and Welleck, 2025 ; Ayoub et al. , 2026 ; Liu et al. , 2026 ; Qi et al. , 2025 ) . Other approaches rely on oversampling and training on shorter responses (Shrivastava et al. , 2025 ) , adopt two-stage training procedures (Song and Zheng, 2025 ; Fatemi et al. , 2025 ) . More recently, several works have also explored training efficient reasoners through self-distillation (Zhao et al. , 2026 ; Sang et al. , 2026 ) . The central challenge is to reduce unnecessary verbosity without making the model overly conservative, which can hinder exploration at test time. These works can be seen as developing efficient low-level solvers that may be leveraged as a part of a bigger system at inference time. 3 TAB: Turn-Adaptive Budgets In this section, we formulate the efficient multi-turn reasoning problem as a sequential compute allocation problem instead of a single-response length-control problem. We model this as a multi-objective Markov decision process and propose TAB: Turn-Adaptive Budgets , a budget allocation policy learned via the GRPO algorithm with a reward function that is a convex combination of the accuracy and adherence to global token budget. 3.1 Problem Setting Notation. In this work, we study multi-turn reasoning under a global per-problem token budget. Consider a large language model f θ ​ ( ⋅ ) f_{\theta}(\cdot) with a set of token budgets ℬ {\mathcal{B}} that generates response y y to query q q by respecting the allotted token budget b ∈ ℬ b\in{\mathcal{B}} as y ∼ f θ ​ ( q ; b ) y\sim f_{\theta}(q;b) (Aggarwal and Welleck, 2025 ) . A multi-turn problem instance x x consists of a sequence of turns, t ∈ [ T ] t\in[T] , with sub-questions q t q_{t} presented to the language model and its responses y t y_{t} as x = ( q 1 , y 1 , q 2 , y 2 , … , q T , y T ) x=(q_{1},y_{1},q_{2},y_{2},\dots,q_{T},y_{T}) where T T is the total number of turns in the conversation episode. Denote the conversation up to turn t t by x 1 : t = ( q 1 , y 1 , … , q t , y t ) x_{1:t}=(q_{1},y_{1},\dots,q_{t},y_{t}) . If at each turn t t , a budget allocation policy π ϕ \pi_{\phi} , parameterized by ϕ \phi , selects how much reasoning compute b t ∈ ℬ b_{t}\in{\mathcal{B}} to allocate to the solver LLM, the response generated can be represented as y t ∼ f θ ​ ( x 1 : t − 1 , q t ; b t ) . y_{t}\sim f_{\theta}(x_{1:t-1},q_{t};b_{t}). Need for Sequential Decision Making in Multi-Turn Systems. Current reasoning models are prone to overthinking and spending far too many tokens on easy turns of a multi-turn conversation leaving little budget for later harder turns. Unlike for single-turn tasks, as demonstrated in Figure 2 , static and context-unaware individual sub-question difficulty based token budget policies are ineffective in the multi-turn setting which present the following set of challenges. First is the temporal dependency and its inherently sequential nature where compute allocation decisions at the current turn made without seeing the future sub-questions determine the future resources remaining, downstream reasoning trajectory and eventual problem accuracy in addition to the quality of the immediate response. Second is the challenge of delayed feedback where rewards presented or accuracy verifiable only at the end of the trajectory resulting in the credit-assignment problem making it difficult to discern which specific turn-level budget decisions contributed to the final outcome. Finally, the problem requires a multi-objective optimization algorithm that can optimize over sequential trajectories to effectively learn the accuracy-tokens tradeoff. 3.2 Multi-Turn Reasoning as a Markov Decision Process We frame the sequential allocation of token budgets as a scalarized multi-objective MDP as follows (Roijers et al. , 2013 ; Van Moffaert and Nowé, 2014 ) . Consider a Markov decision process ℳ {\mathcal{M}} . The state at turn t t represents the problem trajectory so far including all the previous sub-questions and responses and the current sub-question s t = [ x 1 : t − 1 , q t ] s_{t}=[x_{1:t-1},q_{t}] . The action represents the token budget chosen b t ∼ π ϕ ( ⋅ | x 1 : t − 1 , q t ) b_{t}\sim\pi_{\phi}(\cdot|x_{1:t-1,q_{t}}) . The transition to the next state consists of concatenating the response generated by the solver LLM y t y_{t} to the problem trajectory as x 1 : t = ( q 1 , y 1 , … , q t , y t ) x_{1:t}=(q_{1},y_{1},\dots,q_{t},y_{t}) . Consider reasoning tasks with verifiable rewards and represent accuracy of the generated solution as acc ​ ( x ) = 𝟙 y T = y T ⋆ \text{acc}(x)=\mathbb{1}_{y_{T}=y^{\star}_{T}} where y T ⋆ y_{T}^{\star} represents the ground truth label. The objective of the MDP, thus, is to find an optimal budget allocation policy π ϕ ⋆ \pi^{\star}_{\phi} that maximizes the expected accuracy of the reasoning task subject to a global token budget B B modelled as a soft penalty as π ϕ ⋆ = arg ​ max π ⁡ 𝔼 x , π ​ [ acc ​ ( x ) − λ ​ max ⁡ ( 0 , ∑ t = 1 T b t − B ) ] , \pi^{\star}_{\phi}=\operatorname*{arg\,max}_{\pi}\mathbb{E}_{x,\pi}\left[\text{acc}(x)-\lambda\max\left(0,\sum_{t=1}^{T}b_{t}-B\right)\right], (1) where the budget violation penalty weight λ \lambda characterizes the trade-off between accuracy and token efficiency. Alternative reward scalarizations can also be designed to promote concise reasoning, including exact-target, maximum-length, and smoother sigmoid-based penalties (Aggarwal and Welleck, 2025 ; Arora and Zanette, 2025 ; Ayoub et al. , 2026 ) . We use a hinge penalty because our goal is not to match a target length, but to discourage violating a global per-problem budget. This formulation aligns closely with the deployment objective and is less susceptible than naive additive penalties to pathological under-allocation. 3.3 Training Turn-Adaptive Budget Allocation Policy via GRPO Reward Function. We learn the optimal budget allocation policy π ϕ ⋆ ​ ( b t | x 1 : t − 1 , q t ) \pi^{\star}_{\phi}(b_{t}|x_{1:t-1},q_{t}) where b t ∈ ℬ b_{t}\in{\mathcal{B}} using the Group Relative Policy Optimization (GRPO) algorithm (Shao et al. , 2024 ) as follows. Note that we adopt GRPO over Proximal Policy Optimization (PPO) (Schulman et al. , 2017 ) as it eliminates the need for a learned value function, which is notoriously difficult to estimate accurately in long-horizon, sparse-reward settings such as multi-turn reasoning. Now, for each training problem i i , we sample a group of G G multi-turn trajectories { x ( i , g ) } g = 1 G \{x^{(i,g)}\}_{g=1}^{G} . Recall that for every turn t t of a trajectory, the budget allocator observes the current conversation state, [ x 1 : t − 1 ( i , g ) , q t ( i , g ) ] [x_{1:t-1}^{(i,g)},q_{t}^{(i,g)}] , chooses the budget tokens by sampling b t ( i , g ) ∼ π ϕ ( ⋅ | x 1 : t − 1 ( i , g ) , q t ( i , g ) ) b_{t}^{(i,g)}\sim\pi_{\phi}(\cdot|x_{1:t-1}^{(i,g)},q_{t}^{(i,g)}) and supplies it to the LLM, which generates the response y t ( i , g ) ∼ f θ ​ ( x 1 : t − 1 ( i , g ) , q t ( i , g ) ; b t ( i , g ) ) y_{t}^{(i,g)}\sim f_{\theta}(x_{1:t-1}^{(i,g)},q_{t}^{(i,g)};b_{t}^{(i,g)}) . After the final synthesis step, each trajectory receives a single terminal reward r ( i , g ) r^{(i,g)} that scalarizes correctness and token efficiency. We design the reward function 1 1 1 Note that while training, we use the number of tokens actually used by the solver LLM at every turn instead of the number of tokens allotted by the budget policy. to learn the optimal policy ( 1 ) as r ( i , g ) ​ ( x ( i , g ) ) = acc ​ ( x ( i , g ) ) − λ ​ max ⁡ ( 0 , ∑ t = 1 T b t ( i , g ) − B ) r^{(i,g)}\left(x^{(i,g)}\right)=\text{acc}\left(x^{(i,g)}\right)-\lambda\max\left(0,\sum_{t=1}^{T}b_{t}^{(i,g)}-B\right) (2) where acc ​ ( x ) ∈ { 0 , 1 } \text{acc}(x)\in\{0,1\} denotes the correctness of the generated solution, B B is the target global token budget and λ \lambda is a penalty weight. Advantage Estimation and Policy Optimization. Following GRPO, rewards are normalized within each group G G and advantages estimated as A ^ ( i , g ) = r ( i , g ) − μ ( i ) σ ( i ) where μ ( i ) = mean ​ ( { r ( i , g ) } g = 1 G ) , σ ( i ) = std ​ ( { r ( i , g ) } g = 1 G ) . \hat{A}^{(i,g)}=\frac{r^{(i,g)}-\mu^{(i)}}{\sigma^{(i)}}\quad\quad\text{where}\quad\quad\mu^{(i)}=\mathrm{mean}\!\left(\{r^{(i,g)}\}_{g=1}^{G}\right),\sigma^{(i)}=\mathrm{std}\!\left(\{r^{(i,g)}\}_{g=1}^{G}\right). Note that since the reward is terminal and obtained at the end of the trajectory, the same trajectory-level advantage A ^ ( i , g ) \hat{A}^{(i,g)} is assigned to all turns t t in x ( i , g ) x^{(i,g)} . We then maximize a clipped policy objective over all turn-level actions, without a value network as 𝒥 G ​ R ​ P ​ O ​ ( ϕ ) = 𝔼 x ( i , g ) ​ [ 1 | b t ( i , g ) | ​ ∑ k = 1 | b t ( i , g ) | min ⁡ ( ρ t , < k ( i , g ) ​ A ^ ( i , g ) , clip ​ ( ρ t , < k ( i , g ) , 1 − ϵ , 1 + ϵ ) ​ A ^ ( i , g ) ) ] \displaystyle{\mathcal{J}}_{GRPO}(\phi)=\mathbb{E}_{x^{(i,g)}}\left[\frac{1}{|b_{t}^{(i,g)}|}\sum_{k=1}^{|b_{t}^{(i,g)}|}\min\left(\rho^{(i,g)}_{t, Okay, let’s tackle this. The expression is (x + 5)^11 evaluated at x = -2. First, substitute x = -2 into the base: (-2 + 5) = 3. So now it becomes 3^11. Wait, 3^11 is a big number. The simplest form is just 3^11. To simplify $(x + 5)^{11}$ at $x = -2$, substitute $x = -2$: $$(-2 + 5)^{11} = 3^{11}$$ **Answer:** The simplified value is $\boxed{3^{11}}$. LLM-Judge Multi-Turn Trajectory (Budget: 1024 | | Actual: 1,049 tokens): Subquestion: How can we simplify (x + 5)^11 evaluated at x = -2? Response: Okay, let’s tackle this sub-question. The user is asking to simplify (x + 5)^11 when x = -2 ... Let me compute that: -2 + 5 is 3. Expression simplifies to 3^11. Wait, but why is this important? Well, in the context of the main problem, we’re evaluating the entire polynomial at x = -2. But let me double-check... Now, the user might be wondering if there’s a way to compute 3^11 without a calculator... Let me compute it step by step. 3^1 = 3, 3^2 = 9, 3^3 = 27, 3^4 = 81, 3^5 = 243, 3^6 = 729, 3^7 = 2187, 3^8 = 6561, 3^9 = 19683, 3^10 = 59049, 3^11 = 177147. So, 3^11 equals 177,147. However, in the context of the main problem, when we add (x + 5)^11 and (x - 1)^11, which becomes 3^11 + (-3)^11... Since 11 is odd... therefore, they cancel each other out. To simplify the expression $(x + 5)^{11}$... [Continues for another 200 words] ... It’s worth noting that in the context of the main problem, this term combines with $(x - 1)^{11}$... leaving no contribution to the final result. However, this sub-question focuses only on simplifying $(x + 5)^{11}$, not its interaction... **Final Answer:** $\boxed{3^{11}}$ B.2 Effect of Early-Stage Budgets on Later Quality In this example, we demonstrate that the Static baseline with a uniform 512 token allocation for every turn allots too few tokens to a crucial step early in the trajectory leading to compounding errors in subsequent steps and eventually get the answer wrong even though it spends a comparable number of total tokens of 3198 on the problem. In the step shown below, the baseline hits its generation limit mid-thought and results in a wrong algebraic simplification which propagates and results in ’No valid solution.’. On the hand, our policy TAB allots the appropriate number of tokens (1024) to the step shown below, simplifies the equation correctly and in the end solves the question using a total of 3548 tokens. Problem: Solve $\sqrt{x + \sqrt{3x + 6}} + \sqrt{x - \sqrt{3x + 6}} = 6.$ Static (512) Baseline Trajectory (Tokens hit strict cutoff at ∼ \sim 600 tokens) Subquestion: Square both sides of the equation to eliminate outer roots. Response: Let me denote A = sqrt((y^2 + 3y -6)/3) and B = sqrt((y^2 - 3y -6)/3). So the equation is A + B = 6. Squaring both sides gives (A + B)^2 = 36... Now compute A^2 and B^2. A^2 = (y^2 + 3y -6)/3. B^2 = (y^2 - 3y -6)/3. Adding A^2 and B^2: (2y^2 - 12)/3. ... [Generates exactly 653 tokens before hitting the step cutoff limit] ... Expanding the left-hand side: $$ \frac{y^2 + 3y - 6}{3} + \frac{y^2 - 3y - 6}{3} + 2\sqrt{\frac{(y^2 + 3y - 6)(y^2 - 3y - 6)}{9}} = 36 $$ Thus, the equation becomes: $$ \frac{2y^2 - 12}{3} + \frac{2}{3}\sqrt{(y^2 - 9)(y^2 - 6)} = 36 $$ \boxed{\frac{2y^2 - 12}{3} + \frac{2}{3}\sqrt{(y^2 - 9)(y^2 - 6)} = 36} TAB Trajectory (Budget: 1024 | | Actual: 865 tokens) Subquestion: Square both sides of the equation to eliminate outer roots. Response: ... [800+ tokens of rigorous expansion, substitutions, polynomial isolation] ... After squaring both sides, we simplify and derive the equation: $$ y^4 - 21y^2 + 36 = 9\left(400 - \frac{40}{3}y^2 + \frac{1}{9}y^4\right) $$ Simplifying further leads to: 99y^2 - 3564 = 0 \implies y^2 = 36 \implies y = 6 Substituting back, $ y = \sqrt{3x + 6} = 6 \implies 3x + 6 = 36 \implies x = 10 $. Answer: The squared equation simplifies to y^2 = 36, giving x = 10. Appendix C Prompts Recall that User LLM (Qwen3-8B) breaks down the task problem into sub-questions q t q_{t} and presents it to the Solver. The Budgeter LLM (Qwen3-1.7B) takes as input the conversation trajectory and sub-questions and outputs a token budget for every turn as b t ( i ) ∼ π ϕ ( ⋅ | x 1 : t − 1 ( i ) , q t ( i ) ) b_{t}^{(i)}\sim\pi_{\phi}\left(\cdot|x_{1:t-1}^{(i)},q_{t}^{(i)}\right) . The Solver LLM (L1-Qwen3-8B-Exact) that solves the sub-question and generates the response at every turn as y t ( i ) ∼ f θ ​ ( x 1 : t − 1 ( i ) , q t ( i ) ; b t ( i ) ) y_{t}^{(i)}\sim f_{\theta}\left(x_{1:t-1}^{(i)},q_{t}^{(i)};b_{t}^{(i)}\right) . We use the standard chat-based message formats with System, User and Assisstant blocks for all our implementation. It is important to note that the User block in the chat template is different for the User LLM that we use as a component of our Multi-Turn System. Note that the User prompt below is inspired by the Planner prompt in (Lin et al. , 2026 ) . During implementation of the Budgeter LLM, we ask the LLM to predict a difficulty level 0 − 4 0-4 for each sub-question which we then map to the discrete token budget levels as { 0 : 256 , 1 : 512 , 2 : 1024 , 3 : 2048 , 4 : 4096 } \{0:256,1:512,2:1024,3:2048,4:4096\} . User LLM System Prompt: -Goal- You are an experienced expert in math and exam question designer. Your role is to help students break down challenging math problems into a series of simpler, high-level sub-questions. We don’t want too many detailed sub-questions, which are not beneficial for testing students’ ability in an exam. Each sub-question should build on the previous one so that, once all have been answered, the complete solution is clear. Your output should be a list of sub-questions with brief hints explaining the purpose of each step, but you should not reveal your internal chain-of-thought or the final solution. Instructions for Decomposition: First, analyze the problem and identify the key ideas needed to solve it. Then, generate a series of 2 to 5 sub-questions based on the difficulty of question that lead the student step by step to the complete solution. Ideally, we want fewer sub-questions for easy problems and more sub-questions for challenging problems. Do NOT perform reasoning, directly output those sub-questions based on your gut feelings; only output the list of sub-questions with brief hints for each. Your answer should be a list of numbered sub-questions. Each sub-question should have a brief accompanying hint that explains what the student will achieve by answering that part. User Prompt: A student has presented you with the following math problem: Problem: problem **REMEMBER**, you are not allowed to think about it, please directly generate the answer in the following format: Decomposed Sub-questions: Budgeter LLM (TAB) System Prompt: -Goal- You are an expert at assessing the difficulty of mathematical sub-problems within a multi-step problem-solving conversation. You will be given: 1. A math problem 2. The conversation history showing how a solver has worked through previous sub-questions (each turn contains the sub-question and the solver’s answer) 3. The next sub-question that needs a difficulty assessment Your role is to predict the difficulty level (0-4) of the next sub-question, informed by the problem context and the solver’s progress so far. Consider these factors: - How the solver handled previous sub-questions (errors, complexity of responses) - Whether the next sub-question builds on previous answers - The algebraic manipulations and reasoning steps required - The risk of computational errors Format your response as EXACTLY a single number (0, 1, 2, 3, or 4). Nothing else. User Prompt: Here is a math problem being solved step by step: **Problem:** {problem} **Conversation History (previous sub-questions and solver answers):** {conversation-history} **Next Sub-question to rate:** Sub-question {target-idx}: {target-subquestion} Based on the problem, the solver’s progress so far, and the complexity of the next sub-question, rate its difficulty on a scale of 0-4. Output ONLY a single number. Solver LLM System Prompt: You are an expert mathematician solving a complex problem step by step. You will be given sub-questions one at a time with a thinking budget constraint. For each sub-question: 1. Think carefully about what is being asked 2. Use any previous answers and context provided 3. Provide a clear, detailed solution within the token budget 4. State your answer clearly at the end When presenting the final answer to the original problem, you MUST format it as b ​ o ​ x ​ e ​ d ​ { a ​ n ​ s ​ w ​ e ​ r } \\ boxed\{answer\} . Be precise with calculations and show your reasoning within the given token limit. Initial User Prompt: I need help solving the following math problem. I will break it down into sub-questions and ask you one at a time. **Main Problem:** problem Please acknowledge that you understand the problem, and I will present the first sub-question. Initial Assistant Prompt: I understand the problem. I’m ready to solve it step by step within the given thinking budgets. Please present the first sub-question. User Prompt at every turn: **Sub-question idx:** subquestion Please solve this sub-question. Show your work and state your answer clearly. Think for exactly budget tokens. Final Synthesis Prompt: Based on all your work on the sub-questions above, please provide the final answer to the original problem. Present your final answer in the format: b ​ o ​ x ​ e ​ d ​ { a ​ n ​ s ​ w ​ e ​ r } \\ boxed\{answer\} **Final Answer:** BETA