Last night I was reading Michael Bolton’s recent post “Worthwhile Documentation“. This deals with reducing the waste associated with test documentation in order to help deliver fast and effective testing at least cost.
That was shortly after writing yesterday’s post in which I looked at different dimensions we might use to evaluate test approaches. There’s a common thread here that’s worth exploring a little further; the interplay between effectiveness and efficiency in testing.
First, let’s consider some definitions (courtesy of Oxford Dictionaries online):
- Effective: adjective, successful in producing a desired or intended result.
- Efficient: adjective, (of a system or machine) achieving maximum productivity [rate of output per unit of input] with minimum wasted effort or expense.
In business, we often use these terms when we evaluate processes. For example: a production line is effective if it is capable producing the goods that is designed for, it is efficient if it does so with minimal waste (e.g. rejects, wasted materials, energy, time or expense).
How might we relate these to testing activities?
In testing, we test in relation to a mission (for example; delivering information about quality or detecting bugs). Therefore effectiveness is the degree to which our activities are capable of delivering against that mission. We are not generally consuming materials, so efficiency primarily relates to achieving our mission with minimal wasted time or expense.
That sounds simple enough.
Testing however, is not the same as manufacturing: we are not dealing with discrete and quantifiable units, we are not dealing with batches of goods for which production can be considered complete. Instead, we are dealing with experiment and exploration, of reducing uncertainty and revealing information, of a task that can never truly be completed.
This is where things start to get interesting: exhaustive testing is impossible, yet we work within real time and resource constraints. Every test endeavor is therefore a sample. Any waste, any time that doesn’t help to accomplish our mission takes time away from the business of testing, reducing our sample size, reducing our coverage and leaving bugs uncovered.
Is it any wonder that there is such a reaction to waste in testing, as evidenced by Bolton’s post?
In testing, effectiveness and efficiency are inseparable: any reduction in efficiency results in a corresponding loss of effectiveness. Any activity that does not directly contribute to the mission is an opportunity lost.