Testing. We talk about it all the time. I read books, tweets and blog posts debating how to do testing, how to manage testing and usually how to improve my testing. I frequently hear people exclaiming that we’re not doing testing the ‘right way’ or that such and such ‘isn’t really a tester’ but what do we actually mean?
Firstly is seems important to address the question of whether we really need to spend time thinking about this question or not. If you work for a company that has a test team and you have time and resources to test things in the way that you want to, without anyone ever questioning why or how you test then you probably don’t need to spend much time thinking about it. If, however you are trying to grow a test team, or introduce testing more widely to your company then it is worth having a think about.
Personally I think we are wrong to have roles or responsibilities in a team if we don’t understand what we hope to achieve by having them, for that reason alone I think it’s important to understand what you actually mean when you talk about testing.
I think it’s fairly safe to say that the role of a ‘Tester’ will usually involve far more than just executing tests. You’re likely to be working on a test strategy, performing test planning, and risk assessments. You’ll be executing tests, reporting bugs, and maybe even creating automated test suites. The actual act of getting your hands on the system and testing seems to be just one small part of the tester role.
To me testing is about making sure the system does the right thing. I am deliberately vague with those words. Obviously the system must meet all the original requirements but it must also meet many others including, but not limited to, performance, security, accessibility, and reliability. Some of those will hopefully have been specified as part of the actual requirements but I believe testers should be testing them regardless of what was specified. After all is it fair or right to release an insecure product just because the requirements didn’t specify the need to secure it?
The system must also not do unexpected things and this is where I believe the strength of a tester lies; it is impossible to specify all the things that a system must not do and it is also impossible to test that a system doesn’t do all of the things that it shouldn’t do. The tester must assess risks and manage time in order to test a system thoroughly enough for release managers to have enough information be be able to decide if a system is fit for release or not. Good testers use their knowledge of test techniques and risks to break a system in realistic ways. Obviously you might want to to test if the system breaks under unlikely conditions such as a power failure, or DDOS attack but in my opinion there is fairly limited value in testing whether it breaks under near impossible conditions such as if a user attempts to log in 10 times in quick succession. Under tight time constraints I will also focus my testing on scenarios which are likely to occur.
Good testers are able to understand the business goals as well as the user’s goals. They certainly break systems in realistic ways. They bring knowledge of bugs seen in other systems and use the system architecture to quickly identify risk areas. Testers do not have exclusive rights in being able to carry out testing but they do have a particular strength in dedicating their time to studying testing. Anyone who takes the time to read about and practice a skills is likely to be more thorough and have a better understanding of where risk is likely to lie than the causal super user.
The outcome of testing will always be information. Some of that information will be captured in bug reports (things that don’t work as expected), maybe some will be recommendations or feature requests, some will be in completed test plans or mind maps. The outcome of the testing can guide release decisions but it will not provide quality. Quality systems come from understanding the users, the code and making the right decisions about features, infrastructure, code, as well as testing.