Skills and Semantic Layer Enable Self-Service Analytics
Engineering write-up from Anthropic's Data Science & Data Engineering team (Chen Chang, Clement Peng, Justin Leder, Johanne Jiao, Josh Cherry) published on June 3, 2026 on the Anthropic blog (Enterprise AI category, focus on Claude Code). Headline result: "95% of business analytics queries are automated by Claude, with ~95% accuracy in aggregate" (up to ~99% in certain domains). Core problem: analytics is not code — "there's often only a single correct answer using a single correct source" — it requires mapping a user question to precise, up-to-date entities in the data model.
By **Chen Chang// Source claude.com ↗/Reading 2 min/.md// Auto-verified translation
Published on June 3, 2026 on the Anthropic blog, this write-up from the Data Science & Data Engineering team (Chen Chang, Clement Peng, Justin Leder, Johanne Jiao, Josh Cherry) describes how Anthropic made its analytics self-service with Claude: 95% of business queries automated, ~95% accuracy in aggregate (up to ~99% in certain domains).
The starting point is that analytics is not code: "there's often only a single correct answer using a single correct source". The challenge isn't generative creativity but the ability to map a question to precise, up-to-date entities in the data model. Three failure modes threaten this: concept↔entity ambiguity (what counts as "active users"? do fraudsters get excluded? what window?), staleness of assets and the agent's knowledge, and retrieval failure — 80% of failed queries had the information present in the corpus regardless. The worst case is silent failure: a wrong but plausible answer used without objection.
The response is a four-layer"agentic analytics stack". (1) Data foundations: dimensional modeling, canonical datasets"single source-of-truth", metadata treated "as a first-class product", integrity via CI/CD. (2) Sources of truth in decreasing order of trust: a semantic layer the agent is "structurally required (by skill instruction) to leverage first", then lineage, a query corpus distilled into documents (not raw retrieval), and a business knowledge graph (roadmaps, decision logs, org). (3) Skills — the decisive lever: "without skills … didn't exceed 21% … Adding skills gets these numbers consistently above 95%". They are organized in pairs: a router Knowledge skill (~30 reference files) and an Unbook skill encoding the senior analyst's workflow (clarify, find sources, execute, adversarial review). Maintenance is colocated: a review hook flags any model change without a matching skill change — ~90% of data PRs now include a skill in the same diff. (4) Validation: threshold-based offline evals (~90% to authorize an agent), ablation testing, and online guardrails (adversarial review +6% accuracy but +32% tokens and +72% latency; provenance footers; correction harvesting by scheduled agents).
Two negative results shape the doctrine: giving raw grep access to thousands of SQL files moves accuracy "less than a point" (the bottleneck is structure, not access), and letting the LLM define metrics was "net-negative" — hence the rule: documentation generated, definitions owned by humans. To start: a handful of canonical datasets, a few dozen evals, a thin knowledge skill.
Key takeaways
Date / source.June 3, 2026, Anthropic blog (claude.com/blog, Enterprise AI). Authors: Anthropic's Data Science & Data Engineering team (Chang, Peng, Leder, Jiao, Cherry).
Headline result.95% of business analytics queries automated, ~95% accuracy in aggregate (up to ~99% in certain domains).
Thesis. analytics ≠ code — "only a single correct answer using a single correct source" → the challenge is mapping the question to the right, current entities, not generating. ### 3 failure modes 1. Concept↔entity ambiguity — hundreds of options; e.g. "active users" (which actions? exclude fraudsters? which lookback window?). 2. Obsolescence (staleness) — assets and the agent's knowledge become "subtly wrong" (schemas/definitions change continuously). 3. Retrieval failure — 80% of failed queries had the info present in the corpus regardless.
Ultimate danger = silent failure: "the answer is wrong, but looks plausible and is used without objection". ### 4-layer agentic stack
L1 — Data foundations. dimensional modeling, canonical datasets (single source-of-truth, owned, consumption-ready), metadata as a "first-class product", cross-layer integrity via CI/CD.
L2 — Sources of truth. (trust ↓): semantic layer (mandatory first) → lineage/transformation graph → query corpus (distilled into docs, not raw retrieval) → business context (knowledge graph: roadmaps, decision logs, org).
L4 — Validation. offline evals (threshold ~90% to authorize an agent, target ~100%, ground truth fixed on a snapshot, "store results like telemetry"), ablation testing, online (adversarial review, provenance footers, data quality checks, correction harvesting by scheduled agents). ### Notable results (numbers & ablations)
Without skills ≤ 21%. ; with skills > 95% (≈99% in certain domains).
Adversarial review.+6% accuracy, but +32% tokens and +72% latency (a trade-off to accept).
Raw grep ablation. (thousands of accessible SQL files): accuracy moves "less than a point" → the bottleneck is structure, not access.
LLM-generated definitions. = "net-negative" (they encode the very ambiguities meant to be eliminated) → definitions owned by humans.
Unstructured retrieval. across thousands of queries: gain < 1 point → corpus should be distilled, not searched raw.
Maintenance. code-review hook → ~90% of data PRs touch a skill file in the same diff. ### To use in engagements / presentations
Production analytics agent blueprint. that transfers elsewhere: data governance + mandatory semantic layer + skills + threshold evals.
Minimal starting point. (reusable as-is): "a handful of canonical datasets, a few dozen offline evals, and a thin knowledge skill" capture "most of the upside".
Anti-naive-RAG arguments. the two negative results (raw grep, unstructured retrieval) are ammunition against "just give the agent everything".
Living governance metric."90% of PRs include a skill" = proof that docs and code evolve together (anti-staleness).
Connects to: skills (Shihipar/Anthropic), semantic layer/ontology (Talisman, Seale), context engineering (Debois/Tessl), systems around the model (Dropbox), knowledge graph data (Netflix UDA).
Key figures
95% of business analytics queries automated via Claude