Documentation Matters More in the Age of AI, Not Less

Ah, documentation. For most of the history of software development, this was the developer’s least favorite task. And yet, almost all of us have had the painful experience of attempting to understand a poorly-documented codebase. We create design documents, architecture diagrams, process flowcharts, API contracts, code comments, and more. Large enterprise projects often spent weeks or months on planning, design, and alignment because building software was slow and expensive.

Now, the pace of software development has increased. Powerful AI tools are allowing us to work faster and generate enormous amounts of code. One might assume that documentation matters less now because “humans won’t be reading code anymore” and “the AI can figure it out”. This misidentifies why enterprise software development is difficult, and where the actual bottlenecks have moved.

In the enterprise world, the requirements are rarely self-contained. You will encounter problems that no AI tool can solve on its own because it doesn’t know how your company operates. Enterprise software has dependencies, business rules, institutional knowledge, and conventions.

And your code isn’t just for you: it needs to fit within a larger software ecosystem shared by dozens if not hundreds of developers who will read, react to, depend on, and extend your code. Increasingly, coding agents and other AI systems are doing the same.

That brings me to my main idea: documentation is no longer just supplementary information developers add when the code cannot speak for itself. It is increasingly both an input and an output of software development.

Here are four strategies to get more value from documentation in the age of AI.

Make intent explicit

Code tells you what a system does, but often not why. There may be a business rule that seemed obvious when the code was written, an approach that was tried and rejected, or a constraint imposed by another system or team. An AI agent can understand and reasonably extend the existing code, but it may end up being wrong simply because it doesn’t understand the intent.

This is why it is important to make that intent explicit. Before asking an agent to implement a feature, write down what you’re actually trying to accomplish, what constraints need to be preserved, and what behavior you expect. This doesn’t always need to be a long formal design document. Depending on the task, it could be an API contract, architecture diagram, acceptance criteria, or even a short Markdown file describing the desired behavior and important decisions.

I generally like Markdown with embedded Mermaid diagrams for this. It’s easy for both humans and AI agents to read, works well with version control, and doesn’t require special tooling. The specific format matters less than making sure the important decisions and constraints aren’t left implicit.

AI can also help determine whether you’ve made your intent clear enough. Have an agent explain the design back to you, identify ambiguous requirements, poke holes in it, or propose alternatives. If it misunderstands what you meant, there’s a decent chance another developer or agent will too. Better yet, ask for multiple independent critiques, potentially using different models or agent setups. These systems are non-deterministic, and a second or third pass may catch something the first one missed.

But don’t blindly accept its suggestions. An agent may propose a “cleaner” design without understanding the constraint that made you choose the existing approach in the first place. Use AI to challenge your assumptions and improve the specification, but remember that you’re still responsible for deciding what the software should do — and verifying that it actually does it.

Finally, focus your documentation on the things that can’t easily be inferred from the code. Interfaces, contracts, constraints, and the reasoning behind important decisions are generally more useful than explaining implementation details line by line. Sometimes the implementation itself is important — for example, a non-obvious performance workaround or sequencing dependency — and in those cases, document why it needs to stay that way.

Create a reliable source of truth

Modern LLMs are trained on an enormous corpus of data, from social media to academic papers to basically anything else that’s on the internet. By their nature they are generalists, and should be treated as such. You could train your own model or fine-tune an existing open-source one to your particular task. But for most enterprise software problems, you just need a generalist model, a good agentic harness, and the right domain-specific context.

Fortunately, this context should already exist as documentation.

Think about how you collaborate with or onboard a new colleague. You can’t expect them to already know your terminology, data formats, business rules, internal processes, etc. You need to explain those things explicitly, and I believe we should treat AI agents similarly.

When in doubt write it down, even if you aren’t immediately feeding it to an agent.

I generally prefer Markdown because it’s easy for humans, version control, and agents to consume, but the quality and accessibility of the information matter much more than the format.

Some of the most useful sources of context I’ve found are onboarding documents, glossaries, and process documents (domain knowledge as well as company practices).

These can be supplied directly to an agent or exposed through agentic search, MCP servers, retrieval systems, and other tools. I even instruct my agents to propose new glossary entries when a term or acronym is unfamiliar.

In fact, this has an added benefit that I’ll return to later: the agent can reveal issues with the documentation itself, including conflicting information, unstated assumptions, and outdated processes.

Two important caveats:

Sources of truth need to actually be trustworthy. You shouldn’t give an agent incorrect or outdated information, which can be worse than no documentation at all. They can be persistent and stubborn, and if they operate from bad underlying premises, you won’t get great results. Garbage in, garbage out.

There may also be important context that isn’t documented at all and instead exists as institutional knowledge in the heads of SMEs. Identifying and capturing that knowledge can be difficult: SMEs may have limited time, little incentive to document it, or legitimate concerns about how it will be used. Handle this transparently and professionally. Your job as an engineer is to identify missing knowledge and make those gaps visible, not to solve organizational questions about incentives or automation yourself.

Version internal documentation like code

It’s standard practice to version external documentation because APIs, features, and behavior change over time. Internal documentation should be treated the same way.

This becomes especially important in codebases with multiple active branches and environments. Documentation that is accurate for production may be wrong for a feature branch or testing environment.

Keep documentation with the code when its truth changes with the code. Inline comments and docstrings are useful, but often not sufficient. README.md, AGENTS.md, architecture notes, and other documentation should live in version control alongside the code they describe, reducing mismatches.

Give agents access to authoritative operational context when it doesn’t belong in Git. Changelogs, release summaries, internal wikis, deployment systems, and DevOps tools can help an agent understand what has shipped, what is being tested, and what is still under development. Just remember that these sources are only useful if they are trustworthy. I like to run a skill periodically that scans several internal systems and proposes updates to a central wiki, helping keep that operational context in sync.

Manage context deliberately

We’ve thus far discussed what documentation you should make available to your AI agents, but the harness itself makes a huge difference in performance, speed, and cost.

Modern LLMs can have quite large context windows, but that doesn’t mean you should fill them. Performance diminishes well before the limit as the model may overindex on details that don’t matter. Thus it is important to limit context to what’s actually needed.

Searching is different from understanding. Naively one could simply drop thousands or even tens of thousands of tokens into a prompt, and your fancy expensive reasoning agent would bloat its context. Instead, modern agentic harnesses typically separate retrieval from reasoning. One way to do this is to have a search subagent with a smaller model and lightweight tools for shell commands (especially grep and ls), allowing it to narrow down to a relevant passage or file. This avoids context bloat as well as being faster and cheaper.

Know when to get a fresh context window. Especially when switching to a new feature or task, you don’t want to have anything that isn’t needed in the context window. If there are useful details from a previous chat or agent session, it’s generally better to extract and pass only those rather than the entire message history.

Conclusion

As AI makes implementation faster, the bottleneck of software development increasingly becomes specifying intent, providing the right context, and verifying the result. Good documentation supports all three. The developers who get the most value from AI won’t necessarily be the ones who generate the most code — they’ll be the ones who give both humans and agents the clearest understanding of what should be built and why.

Of course, clearly documenting what a system should do is only half the problem. You still need a reliable way to determine whether it actually does it — which is where evals come in. I’ll get into that in a future post.