Skip to content

root / tags / compare-the-market

#Compare the Market

2 fiches

Quality & Security Auto-verified translation

Code review dans le SDLC augmenté : l'anneau de contraintes autour des agents

Episode "Phase 5 · Review" of the SFEIR series on the augmented SDLC, published **the same day** as the Addy Osmani LinkedIn post that it translates into a phase specification. Thesis: **quality has changed address** — it is no longer read in the code (agents produce more of it than anyone can review) but in **the ring of constraints surrounding the agent**. Osmani's ring (seven dimensions — correctness, security, performance, accessibility, maintainability, **economic efficiency**, **comprehensibility** — linked by the **back-pressure** rule: "a loop is only granted the autonomy that can be verified cheaply and reliably, not an inch more") is redrawn, translated, and attached to phase 5 of SFEIR's 11-phase cycle. The structuring corollary: **the bottleneck has never been generation, it is verification** — "generation is a wide mouth, verification a narrow neck; speeding up the mouth thickens the pile at the neck." **The most interesting design decision is a cycle-architecture choice**: Review is deliberately **outside the three human gates** (Define, Plan, Ship), because making Review the gate would put human attention — a finite resource — as the control point of a generation capacity that itself scales: "you would have built a pipeline whose maximum throughput is the number of diffs a senior can read before the end of the day." Hence the split: **Review instruments, Ship decides** — Review delivers an *opposable body of evidence*, Ship decides on the evidence, not on the full diff. A position staked against Monperrus (from whom SFEIR retains the diagnosis — human inspection of every diff cannot withstand agentic speed — but rejects the conclusion: acceptance cannot be delegated). The named trap is **circular validation** (the agent that writes the code writes the tests that validate it: "you built a mirror, not a ring"), with five countermeasures drawn from Anthropic (independent gates in separate context windows, deterministic + agentic never substituting for one another, shadow mode, risk-based tiering, logging to the SIEM) and Compare the Market's warning (**AST graph ~70% vs vector RAG ~58%**, with RAG performing *worse than no context at all*). The firm's own extension is **the ratchet**: "every escape becomes a constraint" — a defect that has crossed the ring is closed *within the ring* (test, lint rule, review rubric, harness guardrail) at Compound-1, "the only asset in the chain that appreciates while the models depreciate" (an unaudited internal measurement: **−30% fix iterations after ten cycles**). It closes by reformulating the question: "is this code good?" has become unanswerable; what remains is **"what does my system refuse to let through?"**

#ring of constraints#constraints around agents#Review phase

SFEIR (voix éditoriale du cabinet, article non signé individuellement) — construit sur Addy Osmani (Google) ; cite Martin Monperrus · Paula Hingel (Augment Code) · DORA/Google Cloud · Jason Clinton (Anthropic) · l'équipe Engineering de Compare the Market

Quality & Security Auto-verified translation

Comparing Context Retrieval Approaches for AI Code Review

Empirical study by the **Compare the Market** engineering team (Meerkat Careers, UK) evaluating four approaches to **context retrieval for AI code review**: Baseline (no additional context), **RAG** (vector search), **GKG** (GitLab Knowledge Graph, AST-based knowledge graph), and **GKG+RAG** (hybrid). Evaluation on **79 real merge requests** with **MLflow on Databricks**. Striking result: **RAG performs worse than the baseline** on almost every metric — vector noise is counterproductive for code review. **GKG outperforms RAG by +21%** in inline comments coverage (0.696 vs 0.577) through structural AST understanding (Tree-sitter + Kuzu graph database). Code requires **structural** understanding (callers, signatures, hierarchies), not mere semantic similarity. GKG costs 4× the baseline but delivers measurable improvements; RAG costs 3× with no improvement. Implemented as a **Docker sidecar** in CI/CD wrapping the GKG binary (still in GitLab beta) with a local MCP server.

#Compare the Market#Meerkat Careers#AI code review

Équipe Engineering Compare the Market (Meerkat Careers, UK — site de comparaison d'assurances et services financiers).