Software testing checks whether software works as intended; quality assurance builds the habits and systems that prevent defects. In modern teams, developers...
Software Testing and QA Engineering in the AI Era: What Changed—and What Still Matters
Software testing checks whether software works as intended; quality assurance builds the habits and systems that prevent defects. In modern teams, developers write many unit and integration tests, while QA engineers focus more on automation, end-to-end workflows, risk, and exploratory testing. AI raises the pace, but human judgment still sets what good means.
For years, I have heard two claims repeated as if one must be true: QA is dead, or QA has never mattered more. Neither helps a team ship reliable software. The real change is that testing work is spread across more roles, and dedicated QA work increasingly focuses on the difficult parts: test infrastructure, full user workflows, risk, and problems that a script cannot anticipate.
I’m Ermenildo Castro, a Seattle-based engineer who has worked at Google, Microsoft, Amazon, and Zulily. Across software teams, I’ve seen a small test suite catch a breaking change before release—and a large one waste everyone’s morning with flaky failures. The difference was not the number of tests. It was whether the checks reflected real user needs and whether people trusted the results. Quality work is a form of stewardship: the software serves people, so reliability and honest trade-offs matter.
What Is Software Testing and QA Engineering?
Software testing is the work of checking whether an application behaves as expected. A test may check one small function, a connection between services, a complete user task, or how a system behaves when traffic rises. Testers compare expected behavior with actual results, then report failures so a team can investigate them.
Quality assurance (QA) is broader. It includes the processes, tools, and team habits that help prevent defects and make product quality more dependable. Testing finds problems; QA also asks why they were introduced, how the team can catch them earlier, and what should change to lower the chance of a repeat.
In everyday job titles, these terms overlap. One company may use QA engineer to mean a person who runs manual test cases. Another may use it for someone who builds automation frameworks, guides test strategy, and works with developers on risk. The title alone tells you little. Ask what the role owns, what it can change, and how the team defines a successful release.
Common forms of testing include:
- Unit testing: Checks a small function or component, often in isolation. The developer who changes the code usually writes or updates these tests.
- Integration testing: Checks whether connected parts work together, such as an application talking to a database or service.
- End-to-end (E2E) testing: Checks a complete workflow through the application, such as signing in and placing an order.
- Regression testing: Re-runs existing checks after a code change to see whether something that used to work has broken.
- Performance and load testing: Checks how the system behaves under expected or unusually heavy traffic.
- Exploratory and manual testing: Uses human judgment to probe for confusing behavior, unusual cases, and issues that scripted tests do not cover.
These test types serve different purposes. A unit test can run quickly but cannot prove that an entire customer workflow works. An E2E test can cover that workflow, but it may be slower and more fragile. The practical test-pyramid idea, discussed in Martin Fowler’s guide to the test pyramid, helps teams think about a balanced mix rather than relying on one layer.
The familiar pattern of writing down exact expected output and comparing it with the actual output also has limits. A normal form, such as a fixed calculation, should give the same answer every time. An AI feature that drafts a response may produce several acceptable answers. Testing that feature means checking whether it stays within clear boundaries, not demanding identical wording. That calls for a test plan built around behavior, safety, and user needs.
Core Benefits and Importance of QA
Good testing reduces avoidable surprises. It does not promise that software will never fail; no serious engineer should promise that. It gives a team faster evidence about what changed, where risk sits, and whether the product is ready for people to use.
- Catch defects closer to the change. A unit test can flag a bad assumption while the author still remembers the code. Fixing it early is usually simpler than tracking it down after several changes have piled up.
- Protect important user tasks. An E2E test can check a workflow people rely on, such as booking an appointment or updating an account. This ties quality work to actual use instead of an abstract count of tests.
- Make releases less of a guess. When a CI pipeline runs useful checks on every change, teams can spot trouble before a release gets bundled with unrelated work. A clear failure is more helpful than a late-night debate about whether the build is safe.
- Reduce repeat work. A regression test can preserve a fix by checking the same behavior in later builds. It becomes part of the team’s memory, especially when the original bug is no longer fresh in anyone’s mind.
- Find performance trouble before users do. Load tests can expose slow responses, exhausted connections, or other bottlenecks under expected demand. These checks matter most when they reflect realistic traffic and system limits.
- Give teams a way to learn. Defects that reach production can reveal a gap in requirements, design, testing, or monitoring. A blameless review should identify the conditions that allowed the failure, then improve the system rather than simply scold the last person who touched the code.
I’ve seen teams with modest automation release more calmly than teams with huge test suites. The calmer teams had a few things in common: someone owned test failures, developers and QA engineers talked before release week, and flaky checks were treated as real maintenance work. Frankly, a test that everyone ignores is not protecting anyone.
QA also protects time. When teams discover a defect early, they avoid repeated debugging, emergency fixes, and avoidable support contacts. That matters to customers and to the people who have to keep the software running. The dignity of the work is not measured by how many defects a team catches; it is seen in careful decisions that respect people’s time, data, and safety.
The numbers need context. Defect escape rate—the defects found after release—is an outcome signal worth watching, but a single number cannot explain why a defect escaped. Test coverage shows which code a test suite exercises; it does not show whether the tests would catch a meaningful failure. Mean time to detect and resolve shows how quickly the team spots and fixes trouble. Flaky test rate shows how often checks fail for reasons unrelated to a real product defect. Use these together, then ask what they say about users and delivery—not just about a dashboard.
Step-by-Step Guide to Building a Modern Testing Approach
Start with risk, not a tool purchase. Teams often buy an automation product before deciding what failure they are trying to prevent. That is backwards. A practical plan connects user needs, code changes, test ownership, and release decisions.
- Name the important user tasks. List the actions that must work for the product to be useful: signing in, saving data, making a payment, or completing a key work task. I start here because a test plan tied to real use is easier to defend than a spreadsheet full of unexplained cases. Keep the list short enough to use.
- Map risk to test layers. Put simple rules and calculations in unit tests, connections between components in integration tests, and a few high-value workflows in E2E tests. Add performance, security, or accessibility checks where the product’s risks call for them. Do not push every check into the slowest layer.
- Set clear ownership. Developers should usually write tests for code they change, especially unit and integration checks. QA or quality engineers can guide test strategy, create shared automation, review coverage of important workflows, and investigate gaps that cross team boundaries. Ownership should be explicit, not settled by whoever has spare time.
- Run the right checks in CI. Start with fast, dependable tests on each change. Add slower suites at suitable points in the pipeline, such as before a release or on a schedule. When a test fails, the result should identify what failed and give someone a reasonable path to reproduce it. A wall of red output is not a useful quality gate.
- Add exploratory testing. A script checks only what someone thought to script. Ask a tester or developer to try unusual inputs, interrupt a workflow, use the product in an unexpected order, and look for confusing feedback. This human pass is especially useful when requirements are unclear or a feature has many paths.
- Treat AI-generated tests as drafts. AI tools can produce a first version of a unit test or E2E script, which may save typing. Review the setup, assertions, edge cases, and failure behavior. Ask whether the test would fail if the product had the defect you care about. If it only repeats the code’s current assumptions, it may add lines without adding protection.
- Review outcomes and repair weak spots. Track production defects, flaky tests, time to detect and resolve, and changes in test performance. When a failure reaches users, review the chain of events: Was the expected behavior clear? Did the test run? Did anyone see the result? I have seen better outcomes when teams fix the missing check and the process that hid it, rather than adding a test and calling the matter closed.
A useful process stays small enough to maintain. Teams should also set a policy for quarantining flaky tests: investigate the failure, give it an owner, and either repair or remove it. Letting unreliable checks fail forever trains people to ignore the pipeline. That is a bad bargain.
Comparison Table: Quality Engineering vs. Developer-Only Testing
The debate is not really between QA and developers. It is about whether quality has a clear owner, whether useful tests run at the right time, and whether someone looks beyond the code being changed. A developer-only approach can work well in a small product with strong engineering habits. It can also leave gaps when nobody owns cross-service workflows, test infrastructure, or exploratory review.
| Area | Dedicated quality engineering | Developer-only testing |
|---|
| Main ownership | QA or quality engineers guide strategy, automation, broader workflows, and exploratory testing alongside developers. | Developers write and maintain most tests as part of code changes. |
| Typical strength | Focuses attention on test infrastructure, user workflows, and risks that span components or teams. | Keeps unit and integration checks close to the code and the person changing it. |
| Common risk | QA can become a late-stage handoff if the rest of the team treats quality as someone else’s job. | Cross-team gaps can persist if developers lack time, skills, or ownership for broader testing. |
| Best fit | Products with complex workflows, high user impact, or many connected systems often benefit from explicit quality engineering work. | Smaller teams may do well when they have clear testing standards and enough time to maintain them. |
| Useful measure | Look at escaped defects, workflow reliability, flaky tests, and how quickly problems are found and fixed. | Use the same outcome measures; code coverage alone does not prove that the tests are effective. |
This is not a contest with one permanent winner. Strong teams combine developer ownership of tests with QA expertise where it adds value. Shift-left testing moves checks earlier, so defects can be found nearer to the code change. It does not mean moving every testing task onto developers or removing dedicated QA roles.
Google’s testing guidance has long warned against making end-to-end testing the only safety net; its discussion of end-to-end tests explains why a heavy reliance on that layer can make suites slow and brittle. A sensible test mix uses smaller, faster checks where they fit, while keeping E2E tests for workflows that matter. Microsoft’s unit testing best practices also give practical advice on writing tests that are focused and useful.
The choice should match the work. A service with a few clear rules may need little dedicated test infrastructure. A product with payments, account access, several services, and strict uptime expectations may need engineers whose job includes quality systems. Frankly, a staffing model is not a quality strategy; clear responsibility and sound tests are.
Common Mistakes to Avoid
Mistake 1: Treating coverage as proof of quality. A high coverage number can look impressive while tests make weak checks or assert only that a function returned something. I once watched a team celebrate a coverage increase, then find that a key error case was never tested. Coverage is useful as a prompt to investigate gaps, not as a score that proves a feature is safe.
Mistake 2: Automating every manual check. Some repeated checks are excellent automation targets. Others are expensive to script and likely to change, or they depend on judgment about whether an interaction feels confusing. Automate stable, important behavior first. Keep human testing for cases where a person can notice what the script cannot describe.
Mistake 3: Writing brittle E2E tests. A test tied to a tiny visual detail can fail after a harmless design update. Prefer stable selectors and assertions about user-visible outcomes, then keep the suite focused on high-value workflows. Self-healing tools may reduce some maintenance, but a test that silently adapts to a broken workflow can hide a real problem.
Mistake 4: Treating flaky failures as normal. A test that passes only sometimes costs more than CI time; it weakens trust in every result. Do not tell the team to rerun failures until they turn green. Find the cause, assign an owner, and make a clear choice to repair, isolate temporarily, or delete the test.
Mistake 5: Adding AI-generated tests without review. A generated test can compile and still check almost nothing. Read the setup and assertions, then imagine the defect the test is supposed to catch. Would the test fail if that defect appeared? If not, it is decoration.
Mistake 6: Making QA a release gate at the end. If developers write code for weeks and hand it to a separate group just before launch, the group inherits a pile of unknowns and a deadline. Share test planning early. Bring QA into discussions about risks, acceptance conditions, and test data before the feature is nearly finished.
Mistake 7: Choosing metrics without the people who use them. A dashboard can track numbers that managers like while developers and testers struggle to explain what those numbers mean. Survey findings on quality gaps have pointed to weak communication and misaligned measures as issues alongside tool problems. Decide together what each metric is for, what it cannot prove, and what action should follow a change.
Most guides say to add more tests. That advice is incomplete. A team needs checks that expose useful failures, a process that deals with them, and enough trust to stop when the evidence says a release is not ready. Here’s the kicker: more automation can make a bad process run faster.
Frequently Asked Questions
Is the QA engineer role disappearing as developers write more tests?
The role is changing more than disappearing. Developers often own unit and integration tests because they are close to the code, while QA and quality engineers increasingly build automation infrastructure, guide test strategy, check E2E workflows, and perform exploratory testing. Some companies still hire for mostly manual test execution, so job duties vary. Read the responsibilities, not just the title. A good QA role should have influence before release week, not merely a queue of finished features to inspect.
Does more AI-generated code mean more bugs in production?
Not by itself. Risk depends on the code, the review process, the tests, and how quickly the team changes its software. AI coding tools can increase the rate of code changes, so a team that does not adjust review and testing may let its checks fall behind. Keep tests and human review strong, inspect generated code, and pay attention to real production outcomes. A faster draft is useful only if the team can judge what it does.
What is the difference between test coverage and test quality?
Coverage estimates how much code a test suite exercises; it does not tell you whether the tests would catch a real defect. Test quality depends on whether the tests check meaningful behavior, cover important edge cases, and fail when the product breaks. A high coverage result can sit beside weak assertions. Pair coverage with escaped defects, flaky test rate, and evidence that tests protect important user tasks.
What is shift-left testing?
Shift-left testing means doing relevant testing earlier in development rather than waiting for a separate testing phase after coding is finished. Developers may run unit and integration checks as part of a code change, and QA can help shape test plans while requirements and design are still being discussed. The aim is to find problems nearer to when they are introduced, when the cause is easier to understand. It is shared work, not a handoff in reverse.
Can AI tools fully automate software testing?
Not for most real-world products. AI tools can draft test cases, help generate scripts, and assist with bug triage, but human review is still important. People must decide what the product should do, whether a generated test checks that behavior, and whether an unusual result is a defect or an acceptable variation. For AI features, teams also need clear limits and meaningful outcome checks, because identical output may not be the right expectation.
Final Thought
The useful question is not whether QA is dead. It is whether a team has the people, time, and tools to learn when its software behaves badly—and to fix the reasons, not just the visible symptom. Developers should own the tests close to their code. Quality engineers should help make broader testing possible, reliable, and tied to actual user needs. Neither group should be asked to carry quality alone.
AI can help draft code and tests, but it cannot decide what a safe release means for a particular product. That takes clear standards, careful review, and a willingness to slow down when the evidence calls for it. In my experience, teams do better when they treat testing as part of building, not as a final inspection. The best measure is not a perfect dashboard. It is software that people can depend on, backed by a team that takes responsibility when it falls short.
— Ermenildo Castro
FAQPage Schema
{"@context":"https://schema.org","@type":"FAQPage","mainEntity":[{"@type":"Question","name":"Is the QA engineer role disappearing as developers write more tests?","acceptedAnswer":{"@type":"Answer","text":"The role is changing more than disappearing. Developers often own unit and integration tests, while QA and quality engineers increasingly build automation infrastructure, guide test strategy, check end-to-end workflows, and perform exploratory testing. Duties vary by company, so read the responsibilities, not just the title."}},{"@type":"Question","name":"Does more AI-generated code mean more bugs in production?","acceptedAnswer":{"@type":"Answer","text":"Not by itself. Risk depends on the code, review process, tests, and rate of change. Teams should keep testing and human review strong, inspect generated code, and monitor production outcomes."}},{"@type":"Question","name":"What is the difference between test coverage and test quality?","acceptedAnswer":{"@type":"Answer","text":"Coverage estimates how much code a test suite exercises; it does not show whether the tests would catch a real defect. Test quality depends on checking meaningful behavior, important edge cases, and failures that matter to users."}},{"@type":"Question","name":"What is shift-left testing?","acceptedAnswer":{"@type":"Answer","text":"Shift-left testing means doing relevant testing earlier in development instead of waiting for a separate phase after coding. Developers may run unit and integration checks with code changes, while QA helps shape test plans earlier."}},{"@type":"Question","name":"Can AI tools fully automate software testing?","acceptedAnswer":{"@type":"Answer","text":"Not for most real-world products. AI can draft tests and scripts, but people still need to define expected behavior, review whether tests check meaningful outcomes, and distinguish defects from acceptable variation."}}]}