Should developers own acceptance tests?

A couple of weeks ago I watched a talk from Dave Farley that said developers should own acceptance tests. It’s a great talk, you should watch it if you haven’t already. Afterwards I realised that I was going to have to write this blog post to explain why I thought the talk was brilliant, but misleading.

First, let’s define acceptance tests. Dave explains them as “trying to assert that the code does what the users want the code to do”. For me they are the tests that we perform (with, or without tools), to decide whether to accept a feature/system or not.

Dave’s talk addresses an important topic. Namely that testers shouldn’t be the owners of these tests. I totally agree with this statement. In all the projects I’ve worked on, I have never seen a test suite solely owned by testers being a useful thing. Of course the testers usually design brilliant tests, and these tests often uncover serious issues before any code gets released but they are still a terrible idea for two reasons.

Firstly, we have an entire suite of tests that exists and runs without any developer involvement. Unless you work in a team that really doesn’t think about testing in any way before code get handed over to testers then this is going to cause test duplication. If your developers really don’t think about testing then you probably have some serious silo issues that need addressing. Go and sort those out before you read the rest of this post. But seriously, developers think about testing all the time. Allowing acceptance testing to be separated from development is going to guarantee duplication in tests. It’s also likely to guarantee some gaps in testing as the approaches of developers and testers are not joined up.

Secondly, testers are brilliant at designing tests. We should expect them to be able to turn up some issues, many of them fundamental and serious. Why wouldn’t we want to do that much earlier in the process? Acceptance tests are usually run against a release candidate. Often a release candidate contains one, or even several weeks of work. Waiting that long to run these tests and then turning up a problem is going to make for some expensive re-work.

In Dave’s talk he argued that developers should be the owners of acceptance tests. To me this sounds like a bad idea. Acceptance tests should be a measure of how well the feature meets the requirements. Developers are probably least qualified, or at least biased enough, to be unable to make this decision. That’s not to say that testers should be the owner either. We still want to avoid creating any kind of “wall” for code to be thrown over.

So who should own the tests? Maybe the team would be a better owner. If acceptance tests are truly going to be a measure of how well the feature meets requirements then it seems to me that a business/product owner needs to decide what gets tested. So we ask the business owner to define the journeys that need testing. Testers know a lot about designing good tests so the Testers should be helping to turn those journeys in to good test scenarios with suitable test data. Developers know how to write robust code so the Developers should be writing and maintaining the tests.

The end result is a well designed, and implemented test suite that actually tests something the product owners wants to have tested.

This collaborate test design brings together the strengths, and input, of the entire team. We give testers the opportunity to uncover issues, but the collaborative nature of the test design means it’s likely to happen far earlier that it would if testers own the tests. Maybe even at the test design stage instead of a week later during release testing. Developers can write the tests and avoid many of the “tester written test” issues. The test scripts get treated as any other code, written and maintained by developers. Finally the real strength of this approach is being able to involve the business owner in the discussions. Hearing about the things they’re worried about right at the beginning can have a massive impact on the design and implementation of the feature.

In fairness I think Dave was actually arguing for exactly this approach. Developers should absolutely be at the heart of acceptance tests but I don’t think we should use the word ‘own’. Teams own things, not individuals. The right group of people collaborate to achieve the best result.

Advertisements

6 comments

  1. Hi Amy,

    thanks for writing this post. I agree with you that all kind of tests should be owned by the team. If the whole team has an open mind to testing, the overall quality of the product will raise, too. In my team, I am writing the test ideas with the help of a mind map, sometimes just bullet points and provide this information to all my team mates. The developer can use this information while coding to see, what kind of mindset I had when writing them down. The feedback I got is great, the team likes this approach and gave them the feeling to catch all those edge cases, too.

    In the end the overall team mind set is really crucial for the quality of the product.
    Keep up the good blogging work :).

    Best and HappyTesting,
    Daniel

  2. Hi Amy, this is a great post and I really like your interpretation and additional information on the original talk.
    I’m currently living in the world of silos you described above, still I kept reading.
    I have a problem with the term “the team owns it”, because usually when everybody owns it, nobody owns it. But your actual advice how to split the responsibilities between roles and make everyone an owner makes really sense and gives another meaning to an otherwise hollow phrase.

    Since I’m currently writing on a test strategy document for our next big project, I will take some time to think about how I could frame your idea into the strategy, to lead the team a bit away from the strong silo thinking.

    Thanks
    Patrick

  3. Great article,

    Whilst I was reading this my immediate thought was that acceptance tests should be owned by the product team and I’m glad that is the end result in this blog.

    As you said I think it’s also good for when the acceptance tests are written up that the devs, testers and product come together to decide what they are before anything is done. This saves time and money

  4. Hi Amy,

    Nice post and I agree your opinion!

    I’m glad that I’m in a team where we don’t have the ownership of tests. Everyone can put his or her thoughts into testing, not only acceptance tests but also others. It’s a common team understanding which I really love. It brings the motivation of team member to involve in and participate testing.

    But as a tester/QA, I do have the responsibility to make sure acceptance tests covering all acceptance criterias. In short word is the responsibility for test design. My other responsibility after acceptance tests during the feature development is to perform exploratory testing, alone or paired, it’s a real fun time for me 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s