# sfeir-zml-llmd-docker-llm-inference-souveraine-2026-07-09

## Veille

SFEIR analysis (consulting-firm voice) of the launch, on July 8, 2026, of **LLMD** by the Paris-based startup **ZML** (founded by **Steeve Morin**, former VP Engineering at Zenly): an inference server that runs LLMs across **five chip families** (NVIDIA CUDA, AMD ROCm, Google TPU, Intel oneAPI, Apple Metal) **from a single codebase**. Structuring thesis: training is ceding the spotlight to **inference**, where cost per token, latency, and above all **dependence on silicon** are now decided. ZML's bet — summed up by the motto *model to metal* — is to **decouple the model from the hardware** via a compiler written in **Zig + MLIR** that produces a hermetic native binary, with no Python in the execution path, exposed through an **OpenAI-compatible API**. Two components, two licenses: **ZML** (the framework, Apache-2.0, >90% Zig) is open source; **LLMD** (the server) is not, free at launch. The article reads the object through three consulting-firm lenses — **token FinOps**, **architectural freedom** (Design to Exit), **sovereignty** (emerging European chips, integration into the VSORA Jotunn8 processor) — then delivers an unsparing verdict: it is an **alpha**, to be placed "under active watch," not to switch to today.

## Titre Article

ZML/LLMD : et si le « Docker des LLM » était français ?

## Date

2026-07-09

## URL

https://www.sfeir.com/articles/zml-llmd-docker-llms-inference-souveraine/

## Keywords

LLM Inference, serving, ZML, LLMD, Steeve Morin, model to metal, Docker for LLMs, model-hardware decoupling, portable not leveled, Zig, MLIR, AOT compilation, hermetic native binary, OpenAI-compatible API, continuous batching, paged attention, prefix caching, tool calling, DFlash, speculative decoding, zero-copy, Hugging Face, NVIDIA CUDA, AMD ROCm, Google TPU, Intel oneAPI, Apple Metal, cold start, AI FinOps, cost per token, cost per outcome, architectural freedom, Design to Exit, reversibility, lock-in, sovereignty, European chips, Axelera, Kalray, SiPearl, VSORA, Jotunn8, Scaleway, Île-de-France Region, VivaTech 2026, Yann LeCun, Solomon Hykes, Clément Delangue, Julien Chaumond, Xavier Niel, vLLM, llama.cpp, technical preview, DGX Spark, Ryzen AI Max+ 395, Apple Silicon, hardware sovereignty

## Authors

SFEIR (voix éditoriale du cabinet)

## Ton

**Profile**: consulting-firm tech analysis (SFEIR thought leadership), aimed at CIOs, architects, and infrastructure decision-makers. Pedagogical, structured register ("In brief," stack layers, "Key points"), high technicality but made accessible, medium length (~1800 words). Clearly situated stance: the firm speaks to its clients and ties the object back to its offering (token FinOps, multi-cloud/multi-hardware, Design to Exit).

**Style**: runs with the **Docker metaphor** ("the *docker run* of inference: one model, one command, any chip") without being fooled by it — the Solomon Hykes investor anecdote serves the figure. A claimed balance between enthusiasm and caution: every vendor promise is **paired with a caveat** ("these figures remain vendor claims, pending independent measurements"; the announced DFlash gain of "up to 10×" is reframed against the "~6.17×" from the underlying research). The "The verdict, unsparing" section adopts an analyst's stance: distinguishes *technical preview* from production, names what the vendor leaves **neither named nor benchmarked** (DGX Spark, Ryzen AI Max+). Systematic house anchoring: refers back to SFEIR articles on sovereign multi-LLM architecture and the multi-cloud vs. sovereign choice, and to the France Télévisions client case (ALIX platform). Explicit, numbered sourcing (zml.ai, TechCrunch, X, arXiv, GlobeNewswire).

## Pense-betes

- **Core idea: inference is the new bottleneck.** After two years focused on model size and training cost, production deployment shifts the focus: what matters day to day is the **cost per token served**, perceived latency, and **dependence on a single silicon vendor**. Serving becomes "both the cloud bill and the strategic room for maneuver."
- **ZML's bet: *model to metal*, decoupling the model from the hardware.** Not yet another model, but an abstraction layer. The stack has **four layers**: (1) models (Qwen, Gemma, Mistral, LLaMa) loaded **zero-copy** via a virtual FS from HF/S3/GCS, with no local download; (2) **LLMD**, a server exposing an **OpenAI-compatible API** (drop-in) with continuous batching, paged attention, prefix caching, tool calling, Prometheus metrics; (3) **ZML**, an **AOT** compiler (Zig + MLIR) into a **hermetic native binary**, with no Python in the execution path; (4) execution on **5 backends**: NVIDIA CUDA, AMD ROCm, Google TPU, Intel oneAPI, Apple Metal.
- **A decisive design choice: "portable, not leveled."** Rather than reducing all accelerators to a lowest common denominator, ZML **preserves chip-specific paths** (FlashAttention on NVIDIA, AITER kernels on AMD). This is what distinguishes a useful abstraction from a performance floor.
- **Figures announced (vendor claims, to be verified).** Very compact container images: ~**1.7 GB** (CUDA), **280 MB** (TPU), **~140 MB** (Apple build). **Cold start** on the order of **1 to 2 s** on an 8B model. The firm stresses: "pending independent measurements."
- **DFlash — built-in speculative decoding.** ZML touts "up to **10×**"; the underlying research (arXiv:2602.06036, *block diffusion for flash speculative decoding*) instead reports **~6.17×** on Qwen3-8B. A nuance worth keeping in front of a technical audience.
- **The two-command demo (Apple Silicon Mac).** `brew install zml/zml/llmd` then `llmd --model=hf://Qwen/Qwen3.6-27B` → a local server with an OpenAI API, queryable via `curl localhost:8000/v1/chat/completions`. The whole value of OpenAI compatibility: scripts/libraries plug back in **with no rewrite**. Two caveats: the first request is slower (graph compilation); a **27B model in BF16** requires **≥ 64 GB of unified memory**.
- **Two components, two licenses (worth keeping in mind before building on top).** **ZML** = open source framework (Apache-2.0, Zig). **LLMD** = **not open source** server, free at launch while **usage data is collected** ahead of future monetization.
- **Three consulting-firm stakes.** (1) **Economic / FinOps**: decoupling the workload from the silicon = the right to choose the cheapest or most available chip → acting on the cost per token (a "cost per outcome" logic). (2) **Architectural freedom**: moving away from "one codebase per hardware target" reduces technical debt and migration cost, and preserves **negotiating power** — this is **Design to Exit** (paying upfront for the layer that makes the vendor replaceable), extended down to the chip level; a parallel with the **France Télévisions / ALIX platform** case. (3) **Sovereignty**: ZML positions itself as an enabler of **emerging European chips** (Axelera, Kalray, SiPearl, VSORA).
- **A concrete sovereignty anchor: VivaTech 2026.** A **ZML × Scaleway × VSORA × Île-de-France Region** partnership around a sovereign inference value chain (from silicon to operation), with the ZML layer integrated into the **VSORA Jotunn8** processor.
- **Credibility through funding.** Founded in Paris in 2023, ~20 people, ~**$20M** raised (20VC, Kima Ventures/Xavier Niel, Kindred Capital, LocalGlobe, Puzzle Ventures). Notable business angels: **Solomon Hykes** (Docker's creator — a nod to the metaphor), **Clément Delangue** & **Julien Chaumond** (Hugging Face), and **Yann LeCun** (Turing), who publicly backs it.
- **Unsparing verdict.** LLMD is a **technical preview**: not for production, with an expanding model catalog. Above all, support for the local machines everyone has in mind (**DGX Spark** ARM64+Blackwell, **Ryzen AI Max+ 395**, recent Macs) **can be inferred** from the backends but is **neither named nor benchmarked** — "to be verified yourself." Competitive positioning: **vLLM** remains the reference for throughput on server GPUs (though Apple Silicon support is community/experimental); **llama.cpp** remains king of single-user local use; LLMD aims for the middle ground — "a vLLM that installs on a Mac." Recommendation: **do not switch, place under active watch.**
- **To cross-reference**: the "sovereignty / multi-cloud vs. sovereign" and "sovereign multi-LLM architecture" family (SFEIR articles); generative AI FinOps and cost per outcome (Didier Girard's AI4IT vs AI4Business note); Steeve Morin already present in the corpus (host of the "À la French" podcast).

## RésuméDe400mots

On July 8, 2026, the Paris-based startup **ZML** released **LLMD**, an inference server that runs large language models across **five chip families** (NVIDIA, AMD, Google, Intel, Apple) from **a single codebase**. SFEIR reads this as a signal: as training cedes the spotlight to **inference**, the real battleground — and cost center — shifts toward **serving**, where cost per token, latency, and dependence on silicon are decided.

ZML's bet comes down to three words, *model to metal*: not offering yet another model, but a layer that **decouples the model from the hardware**. The stack has four layers. At the top, models (Qwen, Gemma, Mistral, LLaMa) loaded **zero-copy** via a virtual file system from Hugging Face, S3, or GCS. Then **LLMD**, a server exposing an **OpenAI-compatible API** (drop-in) with continuous batching, paged attention, prefix caching, tool calling, and Prometheus metrics. Below that, **ZML** compiles the graph **upfront, once and for all**, into a **hermetic native binary** in **Zig + MLIR**, with no Python in the execution path. This binary runs on five backends: CUDA, ROCm, TPU, oneAPI, Metal. The elegance lies in being "portable, not leveled" — chip-specific paths (FlashAttention, AITER) are preserved. Figures announced (by the vendor): images from 1.7 GB (CUDA) to ~140 MB (Apple), cold start of 1-2 s on an 8B model, and the **DFlash** accelerator (claimed "up to 10×," ~6.17× in the underlying research).

Two components, two licenses: **ZML** (the framework) is open source (Apache-2.0, >90% Zig); **LLMD** (the server) is not, free at launch while usage data is collected. The demo runs in two commands on Apple Silicon Macs; a 27B model in BF16 requires ≥ 64 GB of unified memory.

SFEIR reads the object through three client-facing lenses: **FinOps** (choosing the cheapest chip → acting on the cost per token), **architectural freedom** (**Design to Exit**, built-in reversibility, cf. France Télévisions/ALIX) and **sovereignty** (European chips Axelera, Kalray, SiPearl, VSORA; a VivaTech 2026 partnership with Scaleway, VSORA, and the Île-de-France Region, integration into the Jotunn8 processor).

Unsparing verdict: it is an **alpha**, not for production; support for specific local machines (DGX Spark, Ryzen AI Max+) is neither named nor benchmarked. Against vLLM (server-GPU throughput) and llama.cpp (single-user local), LLMD aims for the middle ground. Not to switch to today, but to place "under active watch": a serious, *made in France* candidate to become the "*docker run* of inference."

## GrapheDeConnaissance

- ZML —publie→ LLMD (TECHNOLOGIE, 0.98)
- Steeve Morin —dirige→ ZML (ORGANISATION, 0.95)
- LLMD —permet→ inférence LLM sur cinq familles de puces depuis une seule base de code (AFFIRMATION, 0.95)
- LLMD —utilise→ ZML (TECHNOLOGIE, 0.95)
- ZML —utilise→ Zig (TECHNOLOGIE, 0.95)
- ZML —utilise→ MLIR (TECHNOLOGIE, 0.9)
- LLMD —utilise→ API compatible OpenAI (CONCEPT, 0.95)
- LLMD —utilise→ DFlash (TECHNOLOGIE, 0.9)
- DFlash —améliore→ vitesse de décodage jusqu'à ~6,17× sur Qwen3-8B (MESURE, 0.8)
- model to metal —réduit→ dépendance à un unique fournisseur de silicium (CONCEPT, 0.9)
- LLMD —s_applique_à→ FinOps de l'IA générative (METHODOLOGIE, 0.85)
- LLMD —s_applique_à→ Design to Exit (METHODOLOGIE, 0.85)
- ZML —collabore_avec→ Scaleway (ORGANISATION, 0.9)
- ZML —collabore_avec→ VSORA (ORGANISATION, 0.9)
- ZML —fait_partie_de→ Jotunn8 (TECHNOLOGIE, 0.85)
- Yann LeCun —soutient→ ZML (ORGANISATION, 0.9)
- Solomon Hykes —a_créé→ Docker (TECHNOLOGIE, 0.97)
- LLMD —concurrence→ vLLM (TECHNOLOGIE, 0.85)
- LLMD —concurrence→ llama.cpp (TECHNOLOGIE, 0.85)
- ZML —est_variante_de→ Apache-2.0 (open source) (CONCEPT, 0.9)
- SFEIR —recommande→ mettre LLMD sous surveillance active plutôt que basculer en production aujourd'hui (AFFIRMATION, 0.92)

---
Canonical: https://www.thekb.eu/en/fiches/sfeir-zml-llmd-docker-llm-inference-souveraine-2026-07-09/