Acceptance Tests

Should developers own acceptance tests?

A couple of weeks ago I watched a talk from Dave Farley that said developers should own acceptance tests. It’s a great talk, you should watch it if you haven’t already. Afterwards I realised that I was going to have to write this blog post to explain why I thought the talk was brilliant, but misleading.

First, let’s define acceptance tests. Dave explains them as “trying to assert that the code does what the users want the code to do”. For me they are the tests that we perform (with, or without tools), to decide whether to accept a feature/system or not.

Dave’s talk addresses an important topic. Namely that testers shouldn’t be the owners of these tests. I totally agree with this statement. In all the projects I’ve worked on, I have never seen a test suite solely owned by testers being a useful thing. Of course the testers usually design brilliant tests, and these tests often uncover serious issues before any code gets released but they are still a terrible idea for two reasons.

Firstly, we have an entire suite of tests that exists and runs without any developer involvement. Unless you work in a team that really doesn’t think about testing in any way before code get handed over to testers then this is going to cause test duplication. If your developers really don’t think about testing then you probably have some serious silo issues that need addressing. Go and sort those out before you read the rest of this post. But seriously, developers think about testing all the time. Allowing acceptance testing to be separated from development is going to guarantee duplication in tests. It’s also likely to guarantee some gaps in testing as the approaches of developers and testers are not joined up.

Secondly, testers are brilliant at designing tests. We should expect them to be able to turn up some issues, many of them fundamental and serious. Why wouldn’t we want to do that much earlier in the process? Acceptance tests are usually run against a release candidate. Often a release candidate contains one, or even several weeks of work. Waiting that long to run these tests and then turning up a problem is going to make for some expensive re-work.

In Dave’s talk he argued that developers should be the owners of acceptance tests. To me this sounds like a bad idea. Acceptance tests should be a measure of how well the feature meets the requirements. Developers are probably least qualified, or at least biased enough, to be unable to make this decision. That’s not to say that testers should be the owner either. We still want to avoid creating any kind of “wall” for code to be thrown over.

So who should own the tests? Maybe the team would be a better owner. If acceptance tests are truly going to be a measure of how well the feature meets requirements then it seems to me that a business/product owner needs to decide what gets tested. So we ask the business owner to define the journeys that need testing. Testers know a lot about designing good tests so the Testers should be helping to turn those journeys in to good test scenarios with suitable test data. Developers know how to write robust code so the Developers should be writing and maintaining the tests.

The end result is a well designed, and implemented test suite that actually tests something the product owners wants to have tested.

This collaborate test design brings together the strengths, and input, of the entire team. We give testers the opportunity to uncover issues, but the collaborative nature of the test design means it’s likely to happen far earlier that it would if testers own the tests. Maybe even at the test design stage instead of a week later during release testing. Developers can write the tests and avoid many of the “tester written test” issues. The test scripts get treated as any other code, written and maintained by developers. Finally the real strength of this approach is being able to involve the business owner in the discussions. Hearing about the things they’re worried about right at the beginning can have a massive impact on the design and implementation of the feature.

In fairness I think Dave was actually arguing for exactly this approach. Developers should absolutely be at the heart of acceptance tests but I don’t think we should use the word ‘own’. Teams own things, not individuals. The right group of people collaborate to achieve the best result.

Advertisements