# gao-vercel-agents-md-outperforms-skills-evals-2026-01-27

## Veille

AGENTS.md outperforms skills in Vercel/Next.js agent evaluations

## Titre Article

AGENTS.md outperforms skills in our agent evals

## Date

2026-01-27

## URL

https://vercel.com/blog/agents-md-outperforms-skills-in-our-agent-evals

## Keywords

AGENTS.md, skills, AI agent evaluation, passive context, active retrieval, Next.js 16, Vercel, outdated training data, compressed documentation, system prompt, agentic coding, use cache, connection(), forbidden()

## Authors

Jude Gao (Vercel)

## Ton

**Profile**: Empirical technical post, analytical and factual register, advanced technical level

**Description**: Engineering blog post adopting a rigorous scientific approach with an evaluation protocol, tables of quantified results, and comparative analysis. The tone is that of an engineer sharing experimental results transparently, including failures (skills not invoked in 56% of cases). The argumentation is data-driven with precise percentages. The target audience consists of developers and framework maintainers seeking to optimize their users' experience with AI coding agents.

## Pense-betes

- **Underlying problem**: AI models have outdated training data. Next.js 16 introduces APIs (`use cache`, `connection()`, `forbidden()`) absent from the training data, causing incorrect code
- **Two approaches tested**:
- **Skills**: open standard for packaging knowledge, invocable on demand by the agent
- **AGENTS.md**: static markdown file providing persistent context on every interaction
- **Key results**:
- Without documentation: 53% success rate
- Skills (default behavior): 53% (+0pp, no gain!)
- Skills with explicit instructions: 79% (+26pp)
- **AGENTS.md with index: 100% (+47pp)**
- **Fragility of skills**: in 56% of evaluation cases, the skill was never invoked by the agent
- **Even with explicit instructions**: skill invocation rose to 95%+ but success depended heavily on the exact wording
- **Successful compression**: documentation compressed from 40 KB to 8 KB (80% reduction) with a pipe-delimited format, with no loss of performance
- **Theory of the 3 advantages of passive context**: 1. No decision point (information available without a choice to make) 2. Constant availability (AGENTS.md in the system prompt on every turn) 3. No ordering issues (no action sequencing)
- **Practical recommendations**: do not wait for skills to improve, compress indexes aggressively, build evaluations targeting APIs outside the training data
- **Installation**: `npx @next/codemod@canary agents-md` detects the version, downloads the docs, and injects the index into AGENTS.md
- **Important nuance**: skills remain useful for explicitly triggered specific action workflows, but for general framework knowledge, passive context outperforms

## RésuméDe400mots

Jude Gao from Vercel presents the results of a comparative evaluation between two approaches for conveying framework-specific knowledge to AI agents: skills (active retrieval) and AGENTS.md (passive context). The finding is unambiguous: the simpler method significantly outperforms the more sophisticated one.

**The problem**: Next.js 16 introduces new APIs (`use cache`, `connection()`, `forbidden()`) absent from AI models' training data. Without updated documentation, agents generate incorrect code in 47% of cases.

**Skills: the failure of active retrieval**: Skills, an open standard for packaging knowledge invocable on demand, prove fragile. In 56% of evaluation cases, the agent never invoked the available skill, producing a success rate identical to the baseline (53%). Even when adding explicit instructions forcing invocation (95%+ invocation rate), the success rate only rises to 79%, and results vary dramatically depending on the exact wording of the instructions.

**AGENTS.md: the victory of passive context**: A static markdown file injected into the system prompt on every turn achieves a 100% success rate (build, lint, and tests). This approach eliminates three sources of fragility: the decision point (the agent does not have to choose to invoke), availability issues (the information is always present), and ordering concerns (no action sequencing required).

**Aggressive compression**: The Vercel team managed to compress 40 KB of documentation into an 8 KB index (an 80% reduction) using a pipe-delimited format, while maintaining the 100% success rate. This compression is essential because AGENTS.md consumes context window on every interaction.

**Implications for the ecosystem**: The results suggest that for general framework knowledge, passive context (always available) outperforms active retrieval (on demand). Skills nonetheless retain their usefulness for explicit action workflows. Vercel provides an installation command (`npx @next/codemod@canary agents-md`) that automatically detects the Next.js version and generates the corresponding AGENTS.md.

**Recommendations**: Do not wait for skills to improve, compress documentation aggressively into indexes, and build evaluations that specifically target APIs absent from training data. These results have direct implications for all framework and library maintainers seeking to optimize compatibility with AI coding agents.

## GrapheDeConnaissance

- Jude Gao —publie→ AGENTS.md outperforms skills in our agent evals (DOCUMENT, 0.99)
- Jude Gao —travaille_chez→ Vercel (ORGANISATION, 0.98)
- AGENTS.md —mesure→ taux de réussite 100% (MESURE, 0.99)
- skills —mesure→ taux de réussite 79% (avec instructions explicites) (MESURE, 0.99)
- skills —surpasse→ baseline sans documentation (CONCEPT, 0.97)
- AGENTS.md —surpasse→ skills (TECHNOLOGIE, 0.99)
- contexte passif —réduit→ point de décision agent (CONCEPT, 0.95)
- use cache —fait_partie_de→ Next.js 16 (TECHNOLOGIE, 0.98)
- connection() —fait_partie_de→ Next.js 16 (TECHNOLOGIE, 0.98)
- forbidden() —fait_partie_de→ Next.js 16 (TECHNOLOGIE, 0.98)
- Vercel —a_créé→ Next.js 16 (METHODOLOGIE, 0.97)
- documentation compressée —réduit→ contexte de 40 Ko à 8 Ko (CONCEPT, 0.99)
- skills —mesure→ non-invocation dans 56% des cas d'évaluation (MESURE, 0.99)
- CLAUDE.md —est_variante_de→ AGENTS.md (TECHNOLOGIE, 0.96)

---
Canonical: https://www.thekb.eu/en/fiches/gao-vercel-agents-md-outperforms-skills-evals-2026-01-27/