New Research Proposes Acceleration for Diffusion Language Models

Researchers have introduced On-Policy Residual Learning (ORL), a new method designed to increase the speed of diffusion language models (dLLMs) without compromising generation quality. According to reports shared on the developer community platform Reddit, the research aims to address the inherent inefficiency of diffusion models, which typically require multiple denoising steps to converge on a final output.

The ORL method utilizes two primary technical mechanisms: on-policy residual learning and “Adjacent Entropy Contraction.” Rather than predicting tokens from scratch at every step, the model carries forward previous predictions as residual context. To prevent a mismatch between training and inference, the researchers trained the model on fresh trajectories generated by the model itself. Additionally, the entropy contraction technique encourages each denoising step to reduce uncertainty relative to the previous one, allowing the model to finalize more tokens per forward pass.

Performance Benchmarks and Inference Engine

The researchers released a custom inference engine, dlmrt, optimized for low-latency, single-request decoding. By capturing the entire denoising loop in a CUDA graph, the engine avoids the latency associated with launching steps from the CPU.

Unverified performance claims shared by the developers suggest significant speed increases across several model families:

  • LLaDA-2.1-mini: Reported to reach 2,788 tokens/s on a single NVIDIA RTX 5090 using NVFP4.
  • DiffusionGemma 26B MoE: Reported to achieve 7,708 tokens/s on an 8× B200 configuration.
  • QwenDLM-4B: Reported to see up to a 6.9× increase in end-to-end throughput compared to previous methods.

In comparative testing against autoregressive decoding on a single H100 GPU, the QwenDLM-4B + ORL setup reportedly reached 1,298 tokens/s, compared to 746 tokens/s for DFlash and 405 tokens/s for EAGLE3. The researchers noted these figures represent single-request decoding (batch size 1) rather than aggregate throughput for concurrent requests.

Open Source Release

The research team has open-sourced the models, training code, evaluation pipeline, and the dlmrt inference engine. Model checkpoints have been made available on Hugging Face, including an NVFP4 checkpoint for the LLaDA-2.1-mini model.

While the technical community has expressed interest in the potential for diffusion models in local inference, some observers on Reddit have noted that text diffusion models may currently only offer advantages for single-stream inference, suggesting that commercial incentives for large-scale deployment may remain limited compared to traditional autoregressive models.