I was recently helping a junior tester test an iPhone app. He had found a button which wasn’t doing anything when clicked. The button appeared to click, certainly the graphical interface gave the impression that it was being clicked but nothing happened.
I doubt he is alone in being flummoxed about what to do next. He could have just raised a bug and left it to the developer to investigate but a good tester tries to find as much information as possible before handing it over to someone else.
Here’s the steps we went through to identify the actual cause of the symptom.
Is the problem repeatable? Repeated clicks on the same button led to the same result. So, yes.
Are all buttons affected? We found other buttons responsive. So in this case, no
Does the problem appear to exist for everyone? Testing in a different browser (if web based), a different mobile device (if a mobile app) or even just a different user account can reveal a great deal about the problem. With just a little testing we found the button problem was affecting the user account rather than the device (logging in as the same user on a different device displayed the same behaviour, and logging in on the same device with a different user didn’t).
Now that we know the problem is dependant on user data or settings we can dig deeper. This time I knew the iPhone app was built on top of an API. Directly querying the API displayed the same problem; the number of results being returned didn’t match the API headers which was confusing the iPhone app. This is good news, we have now eliminated the problem from the iPhone app.
We know that different users are experiencing different things. Some have a button that doesn’t respond and others have one that does. We know the problem is in the API or lower rather than in the actual app code. Now we start to investigate everything that could be different for our two identified users.
In this case there are almost no user settings involved so the most likely cause of the problem is data. Identify the unique combinations of data and then eliminate them until you find the trigger. Use a technique such as equivalence partitioning to reduce the number of unique combinations. In this case the problem turned out to be caused by data sets changing ‘state’ without correctly updating all the information in the API.
With just 5 steps we can go from seeing a button apparently not work to identifying unique data groups which trigger the actual problem in a lower layer of the system. If you had just gone ahead and raised a bug about a button not working when clicked, the odds are that it would have worked for the developer. Then the endless ‘Works for me’ cycle begins. So when you see an issue take the time to investigate the actual cause.