# DDTree: Accelerating Speculative Decoding with Block Diffusion Draft Trees

**arXiv**: 2604.12989  
**Published**: 2026-04-14  
**Authors**: Liran Ringel, Yaniv Romano  
**GitHub**: https://github.com/liranringel/ddtree  
**Stars**: 246

## Abstract

Accelerating Speculative Decoding with Block Diffusion Draft Trees

Title:

Content selection saved. Describe the issue below:

Description:

License: arXiv.org perpetual non-exclusive license

arXiv:2604.12989v1 [cs.CL] 14 Apr 2026

Accelerating Speculative Decoding with Block Diffusion Draft Trees

Liran Ringel

1

Yaniv Romano

1,2

Abstract

Speculative decoding accelerates autoregressive language models by using a lightweight drafter to propose multiple future tokens, which the target model then verifies in parallel. DFlash shows that a block diffusion drafter can generate an entire draft block in a single forward pass and achieve state-of-the-art speculative decoding performance, outperforming strong autoregressive drafters such as EAGLE-3. Vanilla DFlash, however, still verifies only a single drafted trajectory per round, potentially limiting its acceptance length. We introduce DDTree (Diffusion Draft Tree), a method that constructs a draft tree directly from the per-position distributions of a block diffusion drafter. Under a fixed node budget, DDTree uses a simple best-first heap algorithm to select the continuations that are most likely to match the target model according to a surrogate defined by the draft model’s output. The resulting tree is verified efficiently in a single target model forward pass using an ancestor-only attention mask. Because DDTree builds on DFlash, a leading draft model for speculative decoding, these gains place DDTree among the leading approaches to speculative decoding.

†

†

footnotetext:

1

Department of Computer Science, Technion – Israel Institute of Technology.

2

Department of Electrical and Computer Engineering, Technion – Israel Institute of Technology.
Correspondence to: Liran Ringel <

liranringel@cs.technion.ac.il

>.

Project Page

Code

Figure 1

:

Speedups relative to autoregressive decoding at temperature 0.0 across datasets and target models. DDTree bars use the best tree-node budget for each dataset-model pair.

1

Introduction

Autoregressive language models generate text one token at a time, so the sampling of each new token requires another forward pass through a large model. This sequential dependence makes decoding a major source of inference latency. Speculative decoding addresses this bottleneck by using a lightweight drafter model to propose several future tokens and a large target model to verify them in parallel, while preserving the target model’s output distribution

[

13

,

5

]

. As language models continue to grow in size, reducing decoding latency without changing model outputs has made speculative decoding an increasingly important technique for inference.

The effectiveness of speculative decoding depends on the quality of the draft model. To deliver end-to-end speedups, the drafter must be cheap enough that drafting overhead stays small and accurate enough that the target model frequently accepts the drafted tokens. At a high level, the objective is to maximize the e

---

Full HTML content extracted from https://arxiv.org/html/2604.12989