Do testers really prevent bugs?

The question of whether testers prevent bugs or find them is a fairly regular tester conversation.

When you find a bug in the software you have clearly found a bug. It makes no difference whether you were actively testing the software, trying to use the software, or simply clicking around randomly, by performing some actions and recognising that a problem occurred you were testing.

Many testers do much more than testing the actual software product and many want to somehow define what these additional things are. Actions like reviewing requirements, planning releases, training users can all fall to testers. So at what point does testing really start?

Richard Bradshaw argues that finding a bug in the requirements is still just finding a bug. You might have found it earlier but you hadn’t prevented it. A mistake in requirements can still be considered a bug. Your knowledge of the system, the domain, or maybe just the user came together to allow you to recognise that there was a problem.

James Bach defines testing as “evaluating the product by learning about it through exploration and experimentation”. You are gathering information about some thing with the intention of using it. There must be a thing, whether a software system, a document, or maybe just an idea about which you can gather data, You must have some intention of using the information to make judgements about the system. If I read the requirements and never stop to consider if there might be gaps, conflicts, or problems with the proposed solution am I testing? Possibly, although I am very unlikely to form any reasonable impressions without trying to use the information I have. When reading through requirements you are most likely to find problems relating to how the system is intended to work if you really imagine what the system is going to do. You have applied your own knowledge to the information and found an issue. In other words you were testing.

So by considering testing to be something that can happen at any time from idea conception right through to general usage in the wild we are saying that testers find bugs rather than prevent them. The bug in the requirements document might not yet exist in the software but it hasn’t been prevented from occurring – it simply occurred in the requirements document rather than the code. The cost of fix a bug at the stage is probably cheaper than if we found it in code but there is still a cost to fixing it.

So, what is a bug anyway?
For me a bug is anything unintentionally undesirable within the system. Testers might uncover things that are undesirable about the system, these things might look like bugs but if we agreed to build the system with some shortcoming on a feature, or not to support a particular browser, these things would not be considered bugs. Suddenly discovering that the system doesn’t scale, or the mobile experience is poor would be bugs. We didn’t plan to make it this way so it is unintentionally undesirable and therefore a bug.

Can bugs be prevented?
I talked about my thoughts on preventing reoccurring bugs in my podcast with Rosie for the Dojo. It’s my experience that systems, and teams tend to re-create the same type of bugs in their work. Maybe there is a particular complexity to your system that trips developers up. Maybe there is some complexity within your domain that leads to repeated re-creation of the same type of bug. Or maybe the experiences of your developers mean they are fantastic with certain things and not so great at others. Whatever the reason I find there is great potential to actually prevent bugs from ever being created by pointing out the re-occurring issues that you see.

Songkick has a special case that we trip over time and time again. Festivals are different from concerts. Both are music events but a festival might not have a venue. It might take place over one day or several. A festival might be announced before any of the line-up are revealed. All of these things can catch us out if we just think about our most common case, the concert.

Rather than waiting to see the issue in requirements, designs, or worse, in the code, I have tried to keep it at the front of everyone’s minds. To me this is bug prevention rather than testing. My techniques involve communication and training rather than creative thinking. This role of sharing knowledge, keeping known areas of risk or complexity visible and talked about does not have to fall to testers. Anyone with some understanding of the system could perform this role. However testers hold valuable knowledge about the system as a whole. They see these bugs reappearing throughout the system (or they’re motivated enough to read through the bug tracking system). Their experience of testing allows them to build a model of the system with interesting features highlighted. Most testers re-use this model to quickly jump to the interesting parts of a system. With some imagination this same model can also be used as a tool for training others.

Richard Bradshaw makes a good case that we don’t prevent car accidents by not driving our car. A prevented bug is impossible to measure because it never existed. Rather than treat this as something that is impossible, and therefore not worth the time or energy I consider the likeliness of the bug appearing to be something I can influence. By taking care to share my knowledge with the whole team I have seen the number of bugs in our products drop.

Whether this is preventing bugs or reducing the chance of them appearing seems like a question of semantics. Call it whatever you like but recognise that it is an important service that testers are well placed to provide.

Share your model of the system. Help developers, and designers understand why you make a bee-line for particular aspects of the system. Help them think about the oft-forgotten areas of a system or domain when planning new features. When I perform this role I’m not hunting for the issue, I’m trying to reduce the likelihood of seeing it in existence. I know what the issue is. My goal now is to make sure everyone else in the development team also knows about this issue.

I believe that testers really can prevent bugs by doing things outside of testing. By taking steps to help the whole development team avoid re-creating the same issues testers can free themselves up to have more time for the really interesting testing.

Advertisements

8 comments

      1. Which means – we should teach developers to only use trivial tools and models to only prevent trivial bugs. So my question is – it it fair “to keep at the front of everyone’s mind” rather than keep them in your own mind and free other minds free of that stuff. Or in other words -maybe too much prevention is too much burden to developer and slows down their developing progress?

  1. Testers are testing – meaning they detect and report “bugs”.
    Im my environment (regulatory industry) we have different names for discrapancies we detect namly: Error, Foult, Failure, Defect and they are not the same.
    May be the most close word/description to bug is defect.
    Testers function/activity to defect prevention, in order to prevent defects you must improve the overall development process ant that is not a tester job.
    This kind of work is done by SQA.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s