Truth is a model (from Neil Gershenfeld)

The most common misunderstanding about software testing is that testers seek and find truth (corollary – testers find faults). They don’t — they make and test models.

Kepler packing Platonic solids to explain the observed motion of planets made pretty good predictions, which were improved by his laws of planetary motion, which were improved by Newton’s laws of motion, which were improved by Einstein’s general relativity. Kepler didn’t become wrong because of Newton being right, just as Newton didn’t then become wrong because of Einstein being right; this succession of models differed in their assumptions, accuracy, and applicability, not their truth.

This is entirely unlike the polarizing battles that define so many areas of life: either my political party, or religion, or lifestyle is right, or yours is, and I believe in mine. The only thing that’s shared is the certainty of infallibility.

Building models is very different from proclaiming truths. It’s a never-ending process of discovery and refinement, not a war to win or destination to reach. Uncertainty is intrinsic to the process of finding out what you don’t know, not a weakness to avoid. Bugs are features — violations of expectations are opportunities to refine them. And decisions are made by evaluating what works better, not by invoking received wisdom.

These are familiar aspects of the work of any scientist, or baby: it’s not possible to learn to talk or walk without babbling or toddling to experiment with language and balance. Babies who keep babbling turn into scientists who formulate and test theories for a living. But it doesn’t require professional training to make mental models — we’re born with those skills. What’s needed is not displacing them with the certainty of absolute truths that inhibit the exploration of ideas. Making sense of anything means making models that can predict outcomes and accommodate observations. Truth is a model.

This post is a copy of Neil Gershenfeld‘s response to the 2011 Edge question.  I replaced the words ‘science’ and ‘scientists’ with ‘software testing’ and ‘testers’ in the first sentence.  I also added the phrase in parenthesis in the first sentence.  You could replace the title and the last sentence with ‘Expected results is a model’.  This should explain why testers don’t/shouldn’t try to determine (or even worse prove) whether the software works.  It also describes that ‘bugs’ are not an indication of (human/developer) failure ( When working with good developers, very few defects are of that category).

The agile movement’s principle of making developers responsible for testing is a good idea.  While a developer can be confident of his technical skills (in programming/computer science), meeting a user’s needs, working with complex technology and large systems requires an empirical approach as followed by scientists.

I wrote an article for Tea Time with Testers with concepts similar to this post – a collection of links from the edge’s 2011 question on how to think about software testing.  If the link doesn’t work, you can go to the archives of Tea Time with Testers (!magazines/galleryPage) and look for the July 2013 issue.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s