Software verification is the work of checking that software behaves as intended, meets its requirements, and does not create unacceptable risks. Coding agents...
Software Verification in the Age of Coding Agents: How to Trust Code You Didn’t Write
Software verification is the work of checking that software behaves as intended, meets its requirements, and does not create unacceptable risks. Coding agents can write code quickly, but speed does not prove correctness; teams still need tests, review, security checks, and clear responsibility for every change.
Key Takeaways
- Treat agent-written code like a change from a fast, unfamiliar contributor: useful, but not exempt from review.
- Define expected behavior before asking an agent to implement it.
- Use tests, static analysis, security checks, and human review together; no single check is enough.
- Keep changes small and traceable so people can understand what changed and why.
- Make a person accountable for merging and operating the result.
Coding agents have moved from suggesting a line or two to changing files, running commands, and responding to test failures. That shift makes verification more important, not less. An agent can produce a convincing patch while misunderstanding a requirement, copying a risky pattern, or overlooking a failure path. The hard question is not whether the code looks polished. It is whether the system does the right thing under ordinary, unusual, and hostile conditions.
I have seen teams mistake a clean diff for a correct one. A tidy patch can still break an old workflow that nobody thought to mention in the prompt. The discipline of verification gives a team a way to check the work rather than trust the confidence of the person—or tool—that produced it. The episode title “Software Verification in the Age of Coding Agents,” featuring Sean Falconer, points directly at that concern. This article focuses on the engineering practices behind the question; it is not a transcript or a claim about what was said in the episode.
What Is Software Verification?
Software verification is the process of checking whether a system, component, or change meets its stated requirements and technical expectations. In plain terms, it asks: Did we build it correctly? That question is related to, but not identical to, validation, which asks whether we built the right thing for the people who will use it. A feature can pass every test written for it and still solve the wrong problem.
Verification is not one test at the end of development. It is a set of checks spread across the work: reviewing requirements, examining code, running automated tests, checking security properties, testing integrations, and watching the software after release. Different checks catch different failures. A type checker may flag an invalid value, but it will not usually tell you that a fee calculation is unfair. A unit test can confirm a function’s output for a chosen input, but it cannot prove that the chosen inputs cover every important case.
Coding agents change who produces a patch, not what correctness means. They can generate a proposed implementation from instructions, repository context, and tool output. Their proposal may be helpful, but it is still a hypothesis about how the program should change. The team must verify that hypothesis against requirements, existing design, and the real operating environment.
In professional engineering work, I have found that the most useful question in a review is often not “Does this code look reasonable?” but “What evidence would convince us that this behavior is safe?” That question leads to concrete checks. For a payment change, evidence might include tests for rounding, retries, duplicate requests, and declined cards. For a permissions change, evidence might include a test proving that a user cannot access another customer’s data.
The distinction matters because agents are good at producing plausible code. Plausibility is not proof. A patch may use familiar libraries and pass a narrow test while still mishandling an empty input, a timeout, a race, or a permission boundary. Verification turns vague confidence into observable evidence. The stronger the impact of a defect, the stronger that evidence should be.
The NIST Secure Software Development Framework describes secure development practices that fit this broader view: prepare the organization, protect software, produce well-secured software, and respond to vulnerabilities. It does not say a coding agent makes those duties disappear. The work remains a human and organizational responsibility.
Why Verification Matters More With Coding Agents
Coding agents can reduce the time it takes to make a first draft. That can help with routine transformations, scaffolding, tests, and exploration of unfamiliar code. But more generated code means more proposed changes to inspect. If review capacity stays fixed while output rises, attention becomes the bottleneck. Teams that celebrate patch volume without protecting review time are measuring the wrong thing.
Here are the practical benefits of a strong verification process:
- Fewer defects reach users. Tests and reviews can catch broken assumptions before deployment. This is especially valuable when a change affects billing, identity, privacy, or data loss.
- Faster diagnosis. Small, well-tested changes are easier to trace when something fails. A large agent-generated patch that mixes unrelated edits turns debugging into archaeology.
- Safer reuse. Agents may copy patterns from nearby code, including old patterns that should not be repeated. Review helps distinguish useful consistency from inherited risk.
- Better team knowledge. Explaining a change and its tests forces engineers to understand how the feature fits the system. The agent cannot take that responsibility on behalf of the team.
- More dependable releases. Clear checks make it easier to decide whether a patch is ready, needs more work, or should be rolled back.
- Stronger accountability. A named person can explain the decision to ship, the known limitations, and the plan if the behavior fails.
In one kind of situation I have met repeatedly in software teams, a small “safe” refactor changes an assumption several services quietly share. The author sees a local improvement. A downstream job sees a changed field, treats it as missing, and stops processing records. Agent tools can make this sort of change quickly, especially if instructions only describe the nearby file. A compatibility test or a search for callers may expose the wider impact before release.
Frankly, “the tests passed” is too thin an argument unless we know what the tests cover. A suite can be green and still be blind to the requirement that matters. Ask which risks the tests address, which cases they omit, and whether the new behavior is observable in production. A green checkmark is evidence, not a blessing.
Verification also protects people who depend on the software. A faulty internal tool may delay a worker’s task; a broken public service may deny access, expose private data, or charge the wrong amount. Engineering choices carry consequences beyond the codebase. Responsible teams treat careful review as part of the dignity of work and as stewardship of systems entrusted to them—not as paperwork to clear before a deadline.
The NIST AI Risk Management Framework is useful here because it emphasizes managing risk across the system’s life cycle, rather than treating AI as a magic feature with a one-time approval. For code-producing tools, that means asking what the agent can access, what it is allowed to change, how its output is checked, and who responds when a problem appears.
A Step-by-Step Verification Process for Agent-Written Code
A good process does not need to be theatrical. It needs to make assumptions visible and failure easier to catch. These steps work for a coding agent, a human contributor, or a mixed team.
- Write down the desired behavior. Before prompting, state what should change, what must stay the same, and how success will be observed. Include important edge cases. “Improve login” is not a requirement. “Lock an account after a defined number of failed attempts, while allowing password reset” is closer.
- Set boundaries for the agent. Give it the relevant files and explain restrictions: do not alter public APIs, do not change database schemas, or do not add dependencies without approval. Limit access to secrets and production systems. An agent should not receive broader permissions just because a task is inconvenient.
- Ask for a plan before a patch. For a risky or unfamiliar change, have the agent list the files it expects to touch, its assumptions, and its proposed tests. Review that outline first. If the plan misses the core requirement, the code will probably miss it too.
- Keep the change small. Split a broad request into reviewable pieces. Smaller patches are easier to compare with the requirement and simpler to revert. In team work, I have seen this save more time than trying to review one giant patch under deadline pressure.
- Inspect the diff, not only the summary. Read the changed lines and check nearby code. Look for unexpected files, weakened validation, broad exception handling, silent changes to configuration, new dependencies, and edits to generated or security-sensitive files. Ask why each change is needed.
- Run layered checks. Start with formatting, linting, type checking, and focused unit tests. Then run integration and end-to-end tests that reflect real use. Add security scanning and dependency checks where appropriate. These checks are complementary; passing one does not cancel out a failure in another.
- Test the edges and the failure paths. Try empty, oversized, malformed, duplicated, delayed, and unauthorized inputs when they apply. Check what happens when an external service times out or returns an unexpected response. Agents often do well on the happy path because that is the easiest behavior to describe.
- Review security and privacy explicitly. Check authentication, authorization, input handling, logging, data retention, and secret exposure. Use the OWASP Top 10 for Large Language Model Applications as a reminder that AI-related systems bring risks such as excessive agency and insecure output handling. The same habit of limiting permissions applies to tools that can modify code or run commands.
- Ask a human to explain the patch. The reviewer should be able to summarize the change, its risks, and the evidence that it works. If nobody can explain why a line is present, pause. “The agent wrote it” is not an engineering rationale.
- Release with a recovery plan. Use staged rollout, monitoring, and a rollback path for changes with meaningful user impact. Watch the metrics tied to the requirement, not only server health. A service can be up while its business behavior is wrong.
I use this order because it catches cheap mistakes early and reserves human attention for questions machines handle poorly: intent, trade-offs, and impact. Teams can adapt the exact tools, but should not skip the reasoning. A practical testing pyramid is one useful model for balancing fast unit checks with slower integration and end-to-end tests. The point is not to chase a fashionable test shape; it is to get fast feedback without pretending one layer covers everything.
Coding Agents vs. Traditional Human-Written Code
Coding agents are not simply a replacement for human contributors. They are another way to produce a proposed change, with different strengths and failure patterns. The comparison below assumes a typical agent-assisted workflow, not a claim that every tool or team behaves identically.
| Area | Coding-agent-generated code | Traditional human-written code |
|---|
| First draft | Often quick for bounded, well-described tasks | Varies with experience and familiarity |
| Context | Depends on prompt, retrieved files, and tool access | Can draw on tacit system and team knowledge |
| Repetition | Useful for boilerplate and consistent transformations | Can be slower and more prone to repetitive typing errors |
| Assumptions | May silently fill gaps in the request | A person may ask questions, though people also guess |
| Review need | Requires careful checking; fluent output can look more certain than it is | Requires careful checking; familiarity can create blind spots |
| Accountability | A human or organization remains responsible for release | The author and organization remain responsible for release |
| Best fit | Bounded tasks with clear rules and strong checks | Ambiguous work requiring judgment, negotiation, or deep context |
The real competitor is not “AI versus people.” It is an unverified change versus a verified change. Human-written code can be wrong, rushed, insecure, or poorly tested. Agent-written code can also be wrong, rushed, insecure, or poorly tested. The source of the patch matters for how you review it, but it does not change the need for evidence.
Agents can be valuable when a task has a clear shape: converting a simple data format, adding repetitive validation, writing a first draft of tests, or locating references across a repository. They are less dependable when the request depends on business rules scattered across documents, unwritten conventions, legal constraints, or sensitive choices about who should have access. A good engineer decides which work is suitable and how much oversight it needs.
Benchmarks can help compare tools on defined tasks, but they do not certify a particular patch for production. The SWE-bench project evaluates systems on software engineering tasks drawn from real repositories; that is useful research context, not a substitute for your own tests, review, and operational checks. A score describes performance on a benchmark set. It does not tell you whether an agent understands your customers, your data contracts, or your failure budget.
Here’s the kicker: choosing an agent does not remove the cost of understanding the code. It moves some of that cost. Time saved on the first draft may return as review, test design, debugging, or maintenance. Measure the whole task—from request through safe release—not just the minutes spent typing.
Common Mistakes to Avoid
The most common failure is not that teams forget to test. It is that they treat a weak check as strong evidence. A few habits make that problem worse.
- Accepting output because it looks polished. Agents can produce neat names, tidy comments, and confident explanations. None of those proves behavior. Read the diff and connect each important change to a requirement.
- Writing tests after the implementation as an afterthought. Tests should challenge the proposed behavior, not merely confirm the code’s current assumptions. Start from the requirement and include failure cases before deciding that the patch is done.
- Using generated tests as the only tests. An agent may share the same mistaken interpretation in both implementation and test. Add independent examples from product rules, prior incidents, or a reviewer’s understanding of user behavior.
- Reviewing only the final message. A summary is a helpful map, not the territory. Check what files changed, what commands ran, whether they completed successfully, and whether errors were ignored.
- Giving an agent excessive access. Broad repository permissions, credentials, or command execution can turn a small task into a large blast radius. Grant only the access needed, and keep secrets away from prompts and logs.
- Treating test coverage as correctness. Coverage can show that code ran; it does not prove that assertions are meaningful. A test that exercises a line but checks nothing important adds little confidence.
- Merging large patches to save time. Big changes are harder to understand and harder to roll back. Split work, especially when it touches authentication, payments, data migration, or public interfaces.
- Skipping production observation. Tests cannot reproduce every real dependency, traffic pattern, or user action. Monitor outcomes after rollout and prepare to stop or reverse a change.
- Blaming the tool instead of fixing the process. Tool failures happen, but teams choose permissions, review standards, and release controls. Responsibility belongs to the people and organization that put the software into use.
I have watched experienced engineers trust familiar code too much, just as newer developers can trust generated code too much. The lesson is not “never trust an agent.” It is “make trust proportional to evidence and impact.” A typo in a private demo and a flaw in access control do not deserve the same review. Risk-based checks let a team spend careful attention where harm would be greatest.
Most guides say to “keep a human in the loop,” then stop there. That phrase is not a process. Name the reviewer, define what they must inspect, preserve the test results, and decide who can approve exceptions. If the human sees only a green badge and a short summary, the loop is mostly decorative.
Frequently Asked Questions
Can coding agents verify their own code?
They can run tests, inspect their output, and suggest fixes. That is useful, but it is not independent verification. If the agent misunderstands the requirement, it may write code and tests that share the same misunderstanding. Use the agent as one checker among several, and have people or independent tools evaluate important requirements. For high-impact changes, include tests based on sources other than the agent’s own explanation.
Are human code reviews still necessary when automated tests pass?
Yes, especially for changes involving security, privacy, money, safety, or complex business rules. Automated tests are strongest when they check known behavior. A reviewer can notice a missing requirement, an unexpected dependency, an overly broad permission, or a design choice that tests do not cover. Reviews should not be ceremonial: focus on intent, risk, and evidence rather than formatting that tools can check.
What should teams verify first in agent-generated code?
Start with whether the patch meets the stated requirement and leaves unrelated behavior alone. Then check the security boundary, error handling, data changes, dependencies, and failure paths. Run tests that match the risk of the change, and inspect whether the agent actually ran them successfully. If the patch affects important user data or access, take extra care with authorization and rollback planning.
How can a team measure whether coding agents are helping?
Measure completed work through safe release, not code volume or the number of generated lines. Track review time, escaped defects, rework, cycle time, reliability, and user impact. Compare similar tasks where possible, and consider maintenance cost after release. A tool that drafts quickly but creates a pile of expensive review work may not be saving time. Keep the measure tied to the outcome the team owes its users.
Final Thought
Coding agents make it easier to produce a patch. They do not make it easier to know whether the patch deserves to ship. That difference is the whole matter. A team that treats generated code as finished work may move quickly for a while, then pay for the missing questions in incidents, rework, and lost trust.
The better approach is plain: define the behavior, limit the tool’s reach, inspect the change, run checks that match the risk, and put a responsible person behind the release. Keep the patch small enough to understand. If the system affects people’s money, privacy, access, or safety, raise the standard of evidence accordingly. That is not distrust for its own sake. It is care for the people who rely on the software.
In my experience, good engineering has never been about writing the most code or adopting the newest tool first. It is about making useful things that work, then being honest when they do not. Coding agents can help with that work when teams give them clear boundaries and keep human judgment where it belongs. Let the tools draft. Let the evidence decide.