Not Checking

Michael Bolton’s distinction between testing and checking seems to have had some unintended consequences. Don’t get me wrong: I have found immense value in this distinction. It’s just that there is some confusion that needs clearing up.

The first confusion relates to value. Some of us (myself included) have fallen into the trap of undervaluing checking. I wrote about this in Checking is Not Evil, and I’m not going to explore that any further today.

The second confusion relates to the relationship between testing and checking. I’ve seen some of this locally, and it is also apparent in some of the comments to Checking is Not Evil. It’s easy to read blog post titles like “Testing vs. Checking” (and possibly reading no further) whilst imaging testing and checking as a Venn diagram that looks like this:






That would be a mistake. As Michael points out in that self-same post, testing involves checking. In other words: checking is something that we can do as part of testing. It is a subset:






The third confusion relates to the content of the-bit-of-testing-that-is-not-checking. One response I’ve run into involves some vague hand waving and allusions to mystical powers that I know about but am not going to share with you. Another is something along the lines of “well obviously that’d be exploratory testing”. There is some truth in this, in that many of the things that make for good ET will fall under this category, but more generally this answer is not a good fit: ET is an approach, a style of testing. It’s definition is too narrow to encompass all of testing less checking.

So what IS the-bit-of-testing-that-is-not-checking? It is what you – as a human being – bring to testing: any skill, any mindset, and any insight that serves your testing mission. You might have skills at identifying checks or automating them, you might have a nose for trouble or a mind made for modeling, you might be a master relationship builder. That-which-is-testing-but-is-not-checking is anything and everything that you can do that a tool cannot, anything and everything that you can bend to the purpose of investigating software so as to provide information about it.

7 thoughts on “Not Checking”

  1. Thanks for writing this – I’ve had to make this same explanation many (many!) times, and will refer people here in the future.

    My only argument with the checking vs. testing thing (which I thought I shared publicly, but I can’t remember if I blogged, tweeted, posted, or where I’ve had the discussion), is that I see checking as /part/ of testing. My sample set is small, but I spend a lot of time explaining how the two concepts are intertwined and related vs. two sides of a testing problem.

    Anyway – thanks. I enjoy reading your blog.

  2. To test you Iain: Is all of checking inside testing – ie a full (math) sub set? Or is there some checking that is not testing 😉

    Also consider: the relative sizes of the sets – that would be context specific right?

    Either way, it’s a illustrative model!

    1. Jesper:

      1) No*, the Death Star is an abstraction. We could refine the model by stating what the universe relates to…but, nah, it’s served its purpose.
      2) Absolutely.
      3) Thanks!

      *One could argue otherwise. Any check has the potential to reveal information (e.g. I check my watch). Depending on how broadly you want to define testing…but now we’re getting silly. 🙂

Leave a Reply

Your email address will not be published. Required fields are marked *