Testers are not unfamiliar with quality: it is central to the information that we seek to discover and share. However, whilst we spend much time considering those characteristics that make up software quality, perhaps we are a little less attentive to the quality of our own testing. Sure, we regularly discuss principles and practices, and whilst these might hold some interest to the tester, these are often seen as mundane matters – especially to our customers. Customers who are often crying out for information, who want their software tested, but have little interest in how it is done. Do you care how electricity is generated and transmitted? About how a power station is designed or how many miles of cable are needed in the distribution network? Or do you simply care that the lights turn on when you flick the switch?
Now, we testers may not “assure” the quality of software, but we had better damn well assure the quality of our own work. Perhaps we should consider what constitutes “quality testing”. So what is “quality testing”? To answer this, let’s turn first to Weinberg: Quality is value to some person. The quality of a product or service is determined by the value it brings to its customers. We need to look to our customers and understand what they value.
I’ve recently been working with some clients to create testing teams, and a desire to understand value has been central to this. This led me to reflect on the value of what we do and how we do it. Or, to put it another way, what characteristics make up the quality of testing?
First, let’s look at the “what” of testing. What we provide is information, that is our raison d’être. But in what ways does that information provide value? Here are some thoughts:
- Relevance. Is the information we provide relevant to the needs of our customers? Does it help them answer the questions that they need to answer?
- Accuracy. Is the information we provide accurate and at an appropriate degree of precision?
- Contemporaneity. Do we provide information at the right time, when our customers need it? Do we provide feedback quickly, contemporaneous with the moment of creation?
Sometimes it is not only the information that we provide that adds value. Our customers sometimes place a value on how we produce information and often the “how” influences or constrains that information. Let’s look at the “how”:
- Transparency. Do we provide transparency, visibility into our work? Do we welcome input by our customers so as to afford them the opportunity to fine tune the value that they will gain?
- Adaptability. Information needs are not static: they evolve throughout the lifecycle of a project, and more generally throughout the lifecycle of a software product. Are we sufficiently adaptable? Do we have the flexibility to change course as we learn more about the software, or as the needs of our customers change? Are we able to take this in our stride or will we fall victim of mission creep? Can we stay relevant?
- Sustainability. As we evolve our service, deepening our understanding of the software and the needs of our customers, are we capable of preserving and sustaining that knowledge? Can we ride out the inevitable changes in team members?
- Compliance. Sometimes our customers have regulatory, audit or standards driven requirements. Can we fulfill the obligations these bring? Can we do so without compromising the value we might otherwise provide?
Of course, our ability to provide value on any of the above dimensions is constrained by time and cost. Tradeoffs have to made, and it is your customers who need to decide which ones. To quote Weinberg again: If you don’t care about quality, you can achieve any other requirement. Do your customers care about the quality of the testing you perform? If so, perhaps a first step to making appropriate tradeoffs is to explore with your customers what quality means to them. I invite you to look inward at the quality of your testing, and to look outward at what your customers value. Let me know what you find.