Quality Isn't a Testing Problem

why fixing your test suite won’t fix your product

I posted this on linkedin last week.

I’ve almost commented on so many posts about what AI is doing to testing.

The short truth is too many people are drawing a false equivalence between testing and quality.

Quality is not a result of testing. Quality is a result of the system you work in. Sometimes testing improves quality. Sometimes it’s actually a hindrance.

Quality problems don’t come from bad or insufficient testing. They come from a system that’s built to produce lower quality.

The responses surprised me, not because people disagreed, but because so many people felt the same way and had been waiting for someone to say it plainly. So let me say it more fully here.

The Equivalence We Keep Making

Here’s how the false equivalence usually works. A product ships with problems. Leadership asks why testing didn’t catch it. The answer is more testing, better testing, earlier testing, automated testing - and most recently AI-powered testing. The assumption underneath all of it is that quality somehow lives in the testing layer and that if we just make that layer thicker or smarter, the problems will go away.

They won’t. More testing has never been the answer.

Testing is an information-gathering activity. It tells you things about the state of your product at a point in time. It doesn’t create quality. It surfaces (some of) the presence or absence of it. If your system is producing low-quality software, better testing will give you a more detailed picture of how low-quality it is. That’s useful information, but it’s nowhere near a solution.

Where Quality Actually Lives

Quality is downstream of everything else. But it’s not primarily downstream of architecture, even though architecture matters. It’s downstream of what your organization reinforces.

I should rephrase. You can have a beautifully designed system and still ship low-quality software if the incentives reward speed over stability, if raising concerns creates friction, if heroics earn recognition while prevention goes unnoticed, or if missed commitments disappear without consequence. The architecture sets the ceiling. The reinforcement determines whether you get anywhere near it.

I wrote about this recently in a post called What Is Your System Teaching You? The core argument is that every organization is running an educational program whether it intends to or not. The question isn’t whether your system teaches, it’s what it’s teaching. If your system teaches people that quality is someone else’s job, that shipping fast matters more than shipping well, or that raising a concern creates more trouble than ignoring it, you will get exactly the quality that curriculum produces. No amount of testing will change that.

This is what I actually mean when I say testing can sometimes be a hindrance. Not that testing is bad. But if testing is positioned as a gatekeeper, and the quality of what’s being shipped is already good enough to deliver real customer value, then that gate is slowing down the customer’s ability to get that value. You haven’t improved anything. You’ve just added latency. The testing didn’t create quality. It delayed delivery while providing the organizational comfort that someone felt they needed.

And if you’re relying on a testing phase to improve quality, you’re already in trouble. A testing phase at the end of a development cycle doesn’t improve the system that produced the problems. It just documents them, hopefully before customers do.

The organizations that get this right don’t treat testing as a phase or a gate. They treat it as a continuous source of signal that feeds back into a system that’s already structured to care about quality. The signal only matters if the reinforcement is there to act on it.

This Isn’t a New Idea

A few years ago, Brent Jensen and I developed the Modern Testing principles with a simple mission statement: accelerate the achievement of shippable quality. The principles that followed were deliberately not about testing in the traditional sense. They were about the whole system.

A few of them are worth calling out here directly. The first principle is that our priority is improving the business. Not improving test coverage, or reducing defect counts. Just improving the business. The third principle says we’re a force for continuous improvement, helping the team adapt and optimize in order to succeed, rather than providing a safety net to catch failures. The fifth principle says the customer is the only one capable of judging and evaluating the quality of our product (testers who think they’re the “voice of the customer” don’t like this one, but good testing heavily leverages customer usage data instead of guessing what customers do).

There’s nothing in there about finding more bugs or about test automation percentages or pass rates. The principles were written by people who had spent years in testing and had come to understand that the value was never in the testing itself. It was in what the testing enabled: better decisions, faster feedback, and a clearer picture of what customers actually experienced.

We called them Modern Testing principles, but as we’ve said many times, they aren’t that modern and they aren’t that much about testing. They’re delivery principles. They’re about the whole team owning quality as a system property rather than delegating it to a function at the end of the process.

What This Means for AI

I’ve watched a lot of hand-wringing about AI replacing testers. Some of it’s warranted. If your testing practice is primarily about executing scripted checks against known expected behaviors, AI will do that faster and cheaper than humans can. That’s not a threat. It’s just the natural end of a particular model of testing that that never worked very well anyway.

AI can’t fix a system that’s built to produce low quality. It can find the symptoms faster, but it can’t treat the disease.

The organizations that’ll benefit most from AI in quality are the ones that already understand quality as a system property. They’ll use AI to get signal faster, to close feedback loops tighter, and to free up humans to do the thinking that actually requires judgment. The organizations that treat AI as a better version of their existing test scripts will get better test scripts and the same quality problems.

The Harder Conversation

The reason this false equivalence persists is that fixing the testing layer is easier than fixing the system. You can hire more QA engineers. You can buy a new tool. You can implement a framework. These are concrete actions with visible outputs, and on the surface, they feel like they matter.

Fixing the system may mean changing how requirements are written, how architects make decisions, or how developers think about their own work.

More importantly, fixing the system requires how leadership sets priorities, and what everyone is measured on. That’s slow, uncomfortable, and hard to put in a quarterly roadmap. So I see teams continue to focus on fixing the testing layer and wondering why the quality problems keep coming back.

Quality isn’t a testing problem. It never was. The sooner we stop treating it like one, the sooner we can start building systems that produce something worth shipping.

---

If this resonates with a problem you’re facing, I have a few consulting slots open. Book some time with me to discuss.