A few days ago, over at http://context-driven-testing.com, Cem Kaner expressed the view that manual exploratory testing had become the new Best Practice. Cem’s comments resonate strongly for me as I have recently recognized and sought to correct that in my own thinking.
Whilst on BBST Test Design last year, Cem called me out on one question where I’d answered from a position of context driven dogma rather than any kind of well thought out position. This was an invaluable lesson that made me begin to question my prejudices when selecting a test approach.
I recognized that I had begun to show a strong preference towards manual exploration over automated testing, of disconfirmatory testing over confirmatory checking. I resolved to challenge myself, and for every approach I considered, to think of at least two others so as to compare and contrast their relative merits.
Then came the moment of truth: a new project. And as luck would have it, one that would go to the very heart of this issue.
The project in question is all about data. There are many thousands of conditions that are specified detail, the data combinations expand this out to potentially billions of individual checks. I would be ignoring a significant part of my testing mission were I to disregard either checking or automation in favor of manual ET. In short, the context demands that large-scale mechanized checking form a big part of the test approach.
Does this mean that we will rely solely on checking and disregard exploration?
Does a check-heavy strategy make this any less context driven?
No, and no
. As I wrote here
, checking may be necessary but it is not sufficient. This project is not unique: there is plenty of opportunity for unfulfilled needs, undesired or unexpected behaviour. We will test the specifications before a single line of executable code is available. We will challenge our own understanding of the design, and seek to align the perspectives and understanding of others
. We will use risk based test design and seek to reveal failures. We will accept that the specification is not perfect, and that we will need to adjust our thinking as the project evolves. We may check, we may not test manually, but the essence of what we do will remain both exploratory and context driven.
Checking is not evil, nor (as far as I am aware) did anyone say that it is.