Muse Spark 1.2 vs Opus 5: Meta's Own Benchmark Results

Key takeaways

1. Date / source. August 5, 2026, Meta AI Research, unsigned announcement. Muse Code (beta), a terminal coding agent installed via curl -fsSL https://dev.meta.ai/install.sh | bash on macOS and Linux, and Muse Spark 1.2, available in Muse Code and in Meta Model API.
2. Key framing. no figures appear in the text — the four comparisons are raster images. The values below were read visually off the PNGs. ### The four comparisons | Comparison | Winner | Muse Spark 1.2 | Rank | |---|---|---|---| | Terminal-Bench 2.1 | Opus 5 + Claude Code — 86.7% | 82.9% | 2nd / 6 | | DeepSWE 1.1 | Opus 5 + Claude Code — 65.0% | 59.3% | 3rd / 6 | | Meta Internal Coding Bench | Opus 5 — 79.4% | 70.6% | 2nd / 5 | | KDA case study (speedup) | Opus 5 — +74.0% | +68.7% | 4th / 6 | On its own proprietary benchmark, Meta sits 8.8 points behind Opus 5; on the case study chosen as a showcase, it also falls behind GPT 5.6 Sol (+71.2%) and behind Anthropic's previous generation, Opus 4.8 (+69.6%). ### The model gain is smaller than displayed On Terminal-Bench and DeepSWE, 1.1 is evaluated with mini-swe-agent and 1.2 with Muse Code: the displayed gap (76.2 → 82.9, i.e. +6.7 points) conflates model progress and harness progress. On the internal benchmark, the only table where no harness is indicated, the 1.1 → 1.2 gap drops to 68.3 → 70.6, i.e. +2.3 points. None of these comparisons is model-against-model: they are model + harness pairs, each model measured with its own lab's harness. This empirically validates, three weeks apart, the mechanism described by [[mozilla-state-of-open-source-ai-2026-07]] — « a harness tuned tightly to one lab's weights becomes a fitted component of that lab's product… Lock-in arrives as a side effect of optimization » — Meta explicitly stating it co-trained the model with the harness « to maximize harness compatibility ». ### The event log « A local event log in which every model call, tool run, approval, and edit is appended », a single source of truth making the runtime « replay-exact and restart-safe »: after a crash, the agent resumes exactly where it left off, which enables long-running tasks. A direct convergence with the run manifest and the run resume command of [[skill-gibbs-hyperresearch-2026-08-03]]. Two independent teams arrive at the same design: context does not survive, an on-disk log does. ### Persistent background agents Instead of being spawned for a task and then destroyed, they « remain active throughout each session », which avoids « redundant information gathering », lets them decide for themselves when to escalate to the main agent, and reduces « latency and the need for steering ». Worth setting against the opposite observation from [[lassiege-usine-logicielle-heure-ia-2026-07-28]] on disposable sub-agents (« I use them less and less »): two opposite answers to the same problem — delegate less, or delegate to agents that no longer die. ### The three shipped skills /plan (a plan submitted for approval), /grill (« stress-tests that plan until it holds up »), /goal. The loop described is plan → challenge → execution with a human gate at the first step. /grill picks up the function of a community skill already documented: a pattern born in the community arrives pre-installed at a frontier lab. ### Self-improvement, read precisely « We also used Muse Spark 1.1 to generate challenging coding environments and instruction-following templates. The model then graded candidate solutions… producing a scalable training dataset for Muse Spark 1.2. » It is 1.1 that generates the environments and does the grading, with 1.2 being the product of that dataset — the subject of « the model » is ambiguous, and some coverage has read it as 1.2. A distillation loop of one generation on itself, whose known limitation is that it can only teach what the previous generation already knows how to evaluate. ### GPU kernel case study: the protocol is worth more than the result Iterative optimization over more than 1,000 cumulative tool calls, up to 24 hours, on the KDA and MLA kernels for NVIDIA Hopper GPUs; the model writes, compiles, profiles, and improves. A notable constraint: « Models were prohibited from importing third-party kernel libraries such as FLA directly » — the algorithm had to be reimplemented in Triton, not wrapped around an existing implementation. This is what makes the test honest. On KDA: a block-parallel preparation kernel plus a sequential inter-block scan, with the cumulative decay re-centered at the block midpoint. On MLA: a two-kernel Triton pipeline reusing the shared KV latent as both K and V, measured at batch size 1, 64 heads, sequence length 8,192, latent dimension 512. KDA is Kimi's linear attention: Meta is thus having its agent optimize the kernel of a competing open-weights architecture. ### Three methodological caveats on the charts 1. No figures in the text: raster images with no text data and no numeric accessibility label. An aggregator, or an agent reading the page, sees no results at all. 2. The KDA chart's y-axis is non-linear, which Meta discloses honestly (« Axis Spaced Linearly In X », tick marks at 0-27-43-53-60-65-69-72-75%). The scale stretches the upper range and visually inflates the gaps between top models — to Meta's disadvantage. 3. The compared configurations are not uniform: max for Opus 5 and GPT 5.6 Terra, high for Grok 4.5 and Gemini 3.6 Flash, nothing for Muse Spark. Grok is absent from the internal benchmark, GPT 5.6 Sol appears only in the case study: the panel's composition changes from one chart to the next. The « Meta Internal Coding Bench » is not public and its composition is not described; Meta measures its competitors on it. The fact that it loses there makes the figure more credible than a flattering score, but a proprietary benchmark remains non-reproducible. ### What the announcement does not say No pricing, no usage limits, no model license, no weight openness — a notable break from the Llama tradition, on which the post is silent. Nothing either on supported languages, context size, or confidentiality guarantees for submitted code.