Risk based testing: a set of test management and design approaches that facilitate the prioritized selection of tests based on ideas concerning possible failures.
Over the last few years, I’ve come across some interesting attitudes towards risk based testing. Here’s three:
- Example 1. Project Manager: “No way! You’re not taking shortcuts on my project!”. This PM thought that we planned on cheating.
- Example 2. Test Manager: “Sure! We do risk based testing: when we hit a crunch we drop the least important tests – we don’t like to do it, but sometimes you have to, right?”. This tester thought that he was cheating.
- Example 3. New Tester: “How do I know what’s important to test?”. Now we’re talking!
In the first two cases, risk based test management was equated with cheating. And it can seem like cheating, right? After all, you can think of a whole bunch of things to test, but many of them wind up being discarded in the interest of time or efficiency.
The reality is closer to the question asked by the naive new tester in example 3. The impossibility of complete testing means we can only ever test a sample: we’re going to test some things and not others. This makes it important to make smart choices about which tests are included in the sample, and which are not: we need to bias our sample towards those tests that are going to yield the most important information.
Risk based testing encompasses a range of approaches for doing just that. It does not hold a monopoly however: almost any testing that isn’t simple grind through a predefined set of tests will involve exactly this kind of thought process, conscious or otherwise.
Whenever a tester asks “should I test this, or that?” some form of risk based decision is being made. These kind of decisions are normal, and they certainly aren’t cheating.