AGENTS.md files define agents, their capabilities, prompts, and safety guards in a repo. They act as a simple, human-readable contract for how an AI assistant...
AGENTS.md Files: New Research Reassesses Their Value for AI Coding
AGENTS.md files define agents, their capabilities, prompts, and safety guards in a repo. They act as a simple, human-readable contract for how an AI assistant should behave, what tools it can call, and which tests must pass before code merges. Useful? Yes, sometimes.
Key Takeaways:
- AGENTS.md is a repository-level doc that stores agent specs, prompts, and safety rules.
- New research questions whether a single markdown file alone is enough for reproducibility and safe deployment.
- Practical value rises when AGENTS.md is paired with tests, CI, and versioned tool schemas.
What is AGENTS.md?
AGENTS.md is a plain-text repository file that documents an AI agent’s role. It lists the agent’s purpose, the tools it may call, sample prompts, and operational constraints. In practice, it reads like a README written specifically for AI: humans and models can both read it, humans solve ambiguity, and the file becomes a source of truth for expected behavior. I first started using such files at a startup where we prototyped automation for data ingestion, and the clarity this small file gave was immediate, it reduced weird behavior, and it stopped junior engineers from shipping dangerous prompts by accident. The file is not a runtime artifact though, and that is the crux of recent research — a doc alone cannot enforce behavior. Researchers compared static, human-readable specs with programmatic interfaces and found gaps in reproducibility, tooling integration, and safety verification. The takeaway is blunt: doc = communication, not enforcement.
Core Benefits and Importance
Clarity is the first win. A small AGENTS.md forces teams to write down expectations, and that reduces confusion about what an agent should do. Traceability follows — you can see which agent did what when you pair the file with logging, and accountability grows. Safety improves because the file can include explicit guardrails, forbidden tool calls, and privacy constraints, and these statements shape code reviews and test design. Reproducibility gets a bump when the doc includes example prompts and canonical inputs; however, reproducibility stalls if there are no tests or if prompts change without version control. I learned this the hard way when a prompt tweak in a private Slack exploded costs, and the AGENTS.md saved our hides because it recorded the last “known-good” prompt. Maintainability becomes easier for onboarding, because a single doc explains how agents work in plain English, and that respects the dignity of work — small acts of stewardship that help teammates succeed.
Step-by-Step Implementation/Guide
1. Create a minimal AGENTS.md first. Write one purpose sentence, list tools, and include one sample prompt.
2. Add constraints and forbidden actions. Be explicit about data handling, and list privacy rules.
3. Put example inputs and expected outputs. Include edge cases and error examples for testers.
4. Version-control the file rigorously. Tag changes in the commit message, and require PR reviews for edits.
5. Link the file to CI checks. Have tests that assert the agent respects tool boundaries, and fail builds on deviant behaviour.
6. Convert parts of AGENTS.md to machine-readable artifacts. Extract tool schemas to JSON or OpenAPI specs so code can enforce them.
7. Add monitoring and post-deploy tests. Run canaries and replay logs against the documented prompts.
I used this exact path at a mid-size company where I led a core developer team, and the pragmatic split between human docs and machine specs worked best. We kept the AGENTS.md short, then generated enforcement code from canonical JSON schemas, and that reduced drift between written intent and executed behavior.
Comparison Table
Below is a side-by-side comparison of a human-readable AGENTS.md file versus a machine-friendly agent_config.json.
| Feature |
AGENTS.md |
agent_config.json |
| Human readability |
High, friendly for reviewers |
Low-medium, needs parsing tools |
| Machine enforcement |
Low, needs adapters |
High, can be validated and enforced |
| Version control clarity |
Medium, diff-able but semantic drift |
High, programmatic diffs possible |
| Testability |
Medium, needs mapping to tests |
High, direct input to CI tests |
| Onboarding speed |
High, plain English |
Medium, needs tools and docs |
| Safety checks |
Medium, relies on reviewers |
High, validator-driven |
Common Mistakes to Avoid
Assuming the doc is enforcement. This is the single biggest error teams make. People write great prose, then expect the model to behave, and that expectation breaks badly when tool APIs change or prompts mutate. Relying on AGENTS.md alone for compliance is naive. Many guides say to pick one canonical place for prompts, but my experience shows that teams who do not formalize schemas will still drift. Forgetting tests is another common blunder. I once audited a repo where the AGENTS.md had perfect examples, but no tests, and production drifted quickly because no CI caught a prompt change. Over-documenting is also a trap. Too many examples create ambiguity, and humans stop updating the doc. Finally, ignoring machine-readable specs isolates the doc. Tooling matters — have both text and schemas.
Frequently Asked Questions
Q: Will AGENTS.md replace programmatic configs?
A: No. The file improves communication, but enforcement needs programmatic artifacts such as JSON schemas, CI hooks, and runtime guards.
Q: Should AGENTS.md live in the main repo?
A: Yes. Keep it in the same repository as the agent code, and require PRs for edits so reviewers can sign off.
Q: How do we test compliance?
A: Create unit tests that run canned prompts against a sandbox model or a stubbed tool interface, assert allowed calls, and fail CI on deviations.
Q: Is this only for research teams?
A: No. Any team building automated agents benefits from explicit documentation and corresponding enforcement.
Files like AGENTS.md are small acts of stewardship for engineering teams. They protect users and teammates by making intent visible, and that matters morally and practically. — Ermenildo V Castro