Tests don't prove anything, but they are a model

I've had a long, love-hate relationship with automated software testing. Testing drove a large part of my career, but boy, have I seen some boondoggles. In my first few years, I learned some critical lessons. The recent talk of AI models has been a little hyperbolic, but it finally gave me a new way to frame what I learned over a decade ago:

We desperately want tests to prove things are correct, but tests are just a model.

The pattern I keep seeing is that software testing makes a great white-whale. After every outage or incident, you can always say "ah, we should have tested that". But that's too late, you needed the test before you shipped the bug to customers.

At every software company, you will find a group of people who've made it their mission to do one of a) just fix the test environment! b) build a test framework! c) get the devs to write more tests! The rare, but spectacular "tell management we're slowing development to write tests!" is a sight to behold.

Sometimes I try to point them in a different direction, sometimes I just watch capitalism do it's thing.

We get older, but the structural incentives never change. Engineers promoted for shipping features, rather than spending weeks writing tests. So... here we are.

I'm using this picture I took of a baby Orca to distract you briefly from the bad news.

The bad news, I'm sorry to tell you is that testing is way more expensive than you thought. Also, your test coverage is likely 0%, and some of the tests you do have, are wrong.

The good news is that once you understand this reality, the recipe for quality software becomes much simpler.

Tests can be wrong, just like models

Can you spot the problem with this code?

def is_invalid_email(input: str) -> bool:
    return input.contains('@')

@Test
def test_is_invalid_email():
    assert is_invalid_email('david@home')
    assert not is_invalid_email('hello')

The problem is that is_invalid_email doesn't do what the name implies, it does the opposite. But the test will pass, because it asserts the wrong behavior.

all models are wrong, but some are useful - George Box

A good reviewer may catch this bug in review. I once had a reviewer notice my test was backwards! The tests didn't save us, but the code-review prevented the bug from being shipped. I call it a win anytime we find a bug before customers do.

Tests are incomplete, because they are a model

Surely it's not that hard to just test all the cases, right? I've heard VPs & managers say things like this, but it's also what your customers want. Customers want you to test for errors before they stumble across them.

An interesting challenge is to try and enumerate all (100%) the test cases for this box. This box is simpler than most real software systems, thus, it has fewer test cases than most real software systems.

Are you done yet? If you think you are, you can check your answer below.

I've been using this box as a motivating example for 10 years now. I've never quite nailed down the computer-science-theory behind the box. Instead, here is one of the greatest proofs of all time:

I shall now show that there is no general method which tells whether a given formula U is provable in K

Alan Turing

On Computable Numbers, with an Application to the Entscheidungsproblem, 1936

Turing's proof would be the greatest mic-drop of all time, if he didn't also follow that up by helping to break Enigma, and coining the term artificial intelligence (AI).

Core computer-science and mathematical realities, like the halting problem, and Turing-completeness make it practically, if not provably impossible to completely test any useful software system.

Tests are expensive, just like models.

I love the SQLite project for many reasons, but one area where it stands out is testing. SQLite is an embedded database, that is almost certainly installed on the device on which you are reading this.

To become so widely deployed, a piece of software has to be trusted by thousands of developers, governments, businesses, and ultimately, users. How does SQLite achieve this? One answer is that they invested heavily in testing.

SQLite has 590 lines of test code for every line of code - https://sqlite.org/testing.html

What would it be like to spend 590x as much time writing tests as we did on the product?

Imagine that you work at BigSoftwareCo. You look around, everybody on your floor is working on test code, the floor above, everybody is also working on test code. Somewhere on the floor above that, there's a single mysterious engineer who gets to actually write the production code that gets shipped to customers. What a dream job!

Of course, much of SQLite's test code is likely automatically generated, but that still means they invested in the test generation code, most of that pre-dates "generative" AI.

So why bother?

It's tempting to dismiss the problems above as a negative anti-testing take. To be fair, my original presentation of this content (below) was a little theatrical with the negativity.

But far from giving up on testing completely, if you were reading closely, I'm showing you how to make testing a powerful, and effective tool. Understanding the problems leads you to the solutions.

1. Tests can be wrong - but they give developers, and reviewers a second chance to find logic errors!
2. You can never achieve 100% coverage - but conversely, Turing-completeness means it's always possible to catch any bug with a (unit) test.
3. Testing can be expensive - but modeling it as a cost-benefit problem allows you to make the right tradeoffs. There's no law saying you have to do have elaborate tests and frameworks. A few unit tests + solid observability is a valid strategy that works for many use-cases.

Tests are a model, and modeling is an extremely powerful tool. Financial analysts use discounted-cash-flow models, physicists have the Newtonian model, and the standard model. Copernicus had the heliocentric model. In software, we are lucky we can create our own models (tests) anytime we want.

What now?

My area of expertise is building software (cloud) services, so here is my prescription for testing software services in 2026.

1. Lots, and lots of unit tests - They are so easy to generate. Even if you think they might be AI-slop, there's still a chance that you (or your tools) will catch a bug by looking at the tests instead of the product code! Take that chance, unit tests are almost free!
2. PROD testing for your critical features - Prod means your production environment, and also means product level testing, a.k.a. end-2-end testing, or testing from the customers perspective. If the customer sends an email, and then expects it to show up on a website, then that's what your test should do. Generating synthetic traffic in your production environment is a very powerful way to improve your observability. Even large-scale services can have a "tree in the forest" problem when things break in the middle of the night. At Amazon, we used to call these canaries.
3. Invest in other strategies too - Accept that tests are just a model, there will be gaps. Then, it becomes more obvious how to pair testing with other strategies. Code-review, monitoring, rollbacks, organizational-learning, linters, type-safety, and progressive rollout strategies are valid & trusted strategies for making your customers happier.

Hopefully this was helpful for you, if you have feedback, you can leave a comment, on LinkedIn.

Published 2026-10-18.

#tech #software #testing #reliability