DiffusionGemma

DiffusionGemma holds 2 billion parameters on a Transformer base derived from «Gemma 2», released early 2025 with open weights published by «Google» on Hugging Face, AI Studio and Vertex AI. «Google» stripped the causal mask, added noise conditioning, and had the model predict distributions over every masked position simultaneously. What comes out is a masked-diffusion generator with bidirectional attention: it starts from a noised sequence and refines it across an adjustable number of denoising steps, and it can update a position it already wrote instead of freezing it.

The MindStudio Team's 12 June 2026 explainer gives the arithmetic that makes the architecture interesting: a 500-token output would take 50 to 100 denoising steps rather than 500 sequential passes. Infilling and constrained generation (templates, code completion with surrounding context, rewriting that preserves a beginning and an end) fall out of the same mechanism. The same piece states the ceiling without hedging. At the 2B scale, DiffusionGemma does not rival GPT-4o or Gemini 1.5 Pro on reasoning, instruction following or knowledge, and the gap closes without being closed. For conversation, multi-step reasoning or token-by-token streaming, autoregressive models stay the better choice.

The second appearance is operational. Michelle Chen, Alex Reneau and Kevin Flansburg report that Cloudflare's «Clef» decision models started from an adaptation of DiffusionGemma that exposed logprobs, resting on work by Matt Mastracci. Shipped «Clef» runs a frozen Qwen3.8-27B. The open weights served the experiment that came first.