Culture

What is Quality?

A few months ago I asked people to complete a short survey on quality. Thanks to all who took part.

Quality is often defined using Jerry Weinberg’s definition of “Value to some person who matters”. Although testers tend to use this definition frequently, and comfortably, I’ve found it to have some limitations. The word ‘value” is almost as fuzzy as “quality” and who exactly are those people who matter? It seems that even with a widely accepted definition it’s still hard to translate quality into a tangible concept for developers or customers.

I decided to try turn a definition of quality into a feeling that people could actually relate to, and hopefully use to better their own work. To do this I asked people to rate the quality of LinkedIn. I know many of you will be laughing at the idea that LinkedIn might represent quality but it turns out to be a very divisive topic. People who want to manage a professional network generally rate LinkedIn’s quality highly. Testing professionals usually don’t.

Although heavily leaning towards the “no”, we can see a decent number of people do think LinkedIn is a quality service.

islinkedinquality

If a service offers value then many people are able to ignore, or maybe don’t even notice shortcomings. I asked people to rate the quality of LinkedIn using just a yes, or a no option. As many respondents pointed out quality is definitely on a scale. What’s interesting to me is that the people who wanted to give a “yes, but no” response tended to say that LinkedIn offers value in managing a professional network (so it is a quality service), but that it it unintuitive to use (so it isn’t a quality service).

The frequency of “intuitiveness” as a measure of quality is of particular interest to the tester side of me. How often do we get caught up testing something against the requirements and ignore the unspecified side of the system? We spend hours hunting down weird and wonderful edge cases, maybe too often choosing to spend our time logging bugs rather than helping to improve the intuitiveness of a system.

We frequently discuss whether testers should learn to code or not. Maybe now is the time to start talking about whether testers should be learning about UX design. At Songkick we see huge benefit in including all team members in testing and quality activities. UI designers are experts at creating usable designs. UX designers understand how users interact with the system. Developers understand how code complexities can cause unexpected behaviour. Ops Engineers know how the system actually runs in production. Product Owners know exactly what they want the system to do. Including all of them in testing discussions brings a wealth of knowledge and skills that, when combined with a testing expert, can have a huge impact on quality.

Quality isn’t the (sole) responsibility of the test team. Everyone involved in developing software should have a good understanding of who the intended users are, their goals, and at least a working understanding of the methods we can use to meet expectations. Facilitate discussions to identify the “people who matter” for your particular system. Help connect team members to users. Even a single session on the customer support desk can have a huge impact on helping people see the real-life impact of a poor design choice. Even just making sure you have a solid approach to managing code maintainability will impact quality as least as much as prioritising features correctly.

Treat quality as a team responsibility. Work together to assist each other in building systems that people actually want to use.

Why I hate DevOps

DevOps. The latest software development fad. Now you can be Agile, use Continuous Delivery, and believe in DevOps.

Continuous Delivery (CD), the act of small, frequent, releases was defined in detail by Jez Humble and Dave Farley in their book – Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation. The approach makes a lot of sense and encourages a number of healthy behaviours in a team. For example, frequent releases more or less have to be small. Small releases are easier to understand, which in turn increases our chances of building good features, but also our chances of testing for the right risks. If you do run into problems during testing then it’s pretty easy to work out the change that caused them, reducing the time to debug and fix issues.

Unfortunately, along with all the good parts of CD we have a slight problem. The book focused on the areas which were considered to be the most broken, and unfortunately that led to the original CD description implying “Done” meant the code was shipped to production. As anyone who has ever worked on software will know, running code in production also requires a fair bit of work.

So, teams started adopting CD but no one was talking about how the Ops team fitted into the release cycle. Everything from knowing when production systems were in trouble, to reliable release systems was just assumed to be fully functional, and unnecessary for explanation.

To try to plug the gap DevOps rose up.

Now, just to make things even more confusing. Dave Farley later said that not talking about Ops was an omission and CD does include the entire development and release cycle, including running in production. So DevOps and CD have some overlap there.

DevOps does take a slightly different angle on the approach than CD. The emphasis for DevOps is on the collaboration rather than the process. Silos should be actively broken down to help developers understand systems well enough to be able to write good, robust and scalable code.

So far so good.

The problem is we now have teams saying they’re doing DevOps. By that they mean is they make small, frequent, releases to production AND the developers are working closely with the Ops team to get things out to production and to keep them running.

Sounds good. So what’s the problem?

Well, the problem is the name. We now have a term “DevOps” to describe the entire build, test, release approach. The problem is when you call something DevOps anyone who doesn’t identify themselves as a dev or as Ops automatically assumes they’re not part of the process.

Seriously, go and ask your designers what they think of DevOps. Or how about your testers. Or Product Managers. Or Customer Support.

And that’s a problem.

We’ve managed to take something that is completely dependant on collaboration, and trust, and name it in a way that excludes a significant number of people. All of the name suggestions that arise when you mention this are just ridiculous. DevTestOps? BusinessDevTestOps? DesignDevOps? Aside from just being stupid names these continue to exclude anyone who doesn’t have these words in their title.

So do I hate DevOps? Well no, not the practice. I think we should always be thinking about how things will actually work in production. We need an Ops team to help us do that so it makes total sense to have them involved in the process. Just take care with that name.

Is there a solution? Well, in my mind we’re still talking about collaboration above all else. Thinking about CD as “Delivery on demand” also makes more sense to me. We, the whole team, should be ready to deliver working software to the customer when they want it. By being aware of the confusion, and exclusion that some of these names create we can hopefully bring everyone into the project before it’s too late.

What works for me might not work for you. Reverse. Repeat.

Earlier this week I posted my thoughts about DevOps and Continuous Delivery. TheSunshine tester responded on Twitter with a strong defense for Agile

I love that methodologies are able to create such passionate responses. Even amongst the truly committed I doubt I would find any passion for the Waterfall method. Yet the Agile, DevOps, and CD communities seem genuinely excited about working together in this way.

Software Development has greatly improved in the last few years but I still think we have a long way to go before it is considered reliable. Last year CIO.com reported that the majority of ERP projects are over time and over budget. If Agile is truly the silver bullet then all teams would be adopting it, and finding success with it. Unfortunately that doesn’t appear to be the case.

No matter how successful Agile, or any practice might prove to be we should still be seeking new and better ways of working. Humans are unique and our involement makes every project equally unique. The team who has success with agile is unlikely to have the same team make-up, or collective experience as one who doesn’t transition, or who fails to succeed with agile. Our interpretation of what a process involves, and our previous experiences all serve to influence what the end process will actually look like.

We need to have discussions about DevOps because it has arisen from an apparent divide between developers and Ops people. Whether this divide is caused by agile, or a failing of the implementation is irrelevant, the fact is that many teams are finding things as they stand are not good enough.

At Songkick we’ve followed Lean practices for many years. We have small cross-functional teams, daily stand-up meetings and weekly retrospectives. We measure things and try to remain flexible and responsive. Despite this we were having trouble releasing working software. Implementing Continuous Deployment was an enhancement to our agile practices. It went beyond giving us a framework for working together and gave us the steps we needed to make frequent releases too.

One of the criticisms I often hear is ‘it wouldn’t work here’. Hopefully these reservations come from understanding that there are no best practices, but whatever your reason for resisting copy-book change you are right.

What has worked for me, in my team, on my project will never work in exactly this way with your team.

Simply put, best practices do not exist.

Continuous Delivery is the current trendy approach and that is a concern. You should be trying to fix a problem not copy a textbook. If you cannot identify the benefit of any practice, agile ones included, then you shouldn’t be using it. One of the most valuable skills we learned from adopting Continuous Delivery was the ability to review our process and identify exactly what was working and what wasn’t. We continue to review things, making changes as our needs, or the team, changes.

Talk to other teams, listen to their experiences, read widely but use their processes as your guide. The most useful thing you can learn from other teams is how they implemented change. What were the triggers or warning signs? How did they know it was working? How did they choose this change?

These are things that you can apply to your own situation. These are the things that are likely to actually help you deliver good software.

DevOps with Testers

Last week I spoke at QCon London about Songkick’s experience of moving to Continuous Delivery. You can download the slides, or watch the longer version of this talk on the LondonCD Meetup’s Vimeo channel.

My talk was part of the DevOps and Continuous Delivery track hosted by Eoin Woods. QCon videos of all the talks should be available soon.

One particularly interesting talk was by Steve Thair on ‘DevOps and the need for speed‘. Previously I thought DevOps was about developers and Ops people collaborating to enable faster software delivery. Steve’s definition went much further and ended up being closer to how I think about Continuous Delivery.

Steve defined DevOps as a model that “encompasses a product-centric view across the entire product life-cycle (from inception to retirement) and recognises the value in close collaboration, experimentation and rapid feedback.” He emphasised that the business and testers should be engaging in the collaboration just as much as developers and ops people.

An interesting distinction between Steve’s DevOps definition and Dave Farley’s Continuous Delivery definition focused on the definition of ‘Done’. Continuous Delivery is defined as being about “making businesses more experimental through the early and continuous release of valuable software”. It appears to be complete once the release is out in production. DevOps adds an important final step of evaluating how much value the release is actually adding once it’s in the hands of users.

I love Steve’s definition of DevOps but unfortunately think this term is sadly restricting. I doubt business people or testers will feel included in something called ‘DevOps’. Even more confusingly I believe there are both devs and ops people who do believe ‘DevOps’ only covers devs and ops.

Even re-defining Continuous Delivery to extend beyond the release doesn’t really help the situation. We’re trying to create a process that keeps the software in a state where by it can be released whenever the business wants it to be. Obviously there are some questions around why the business would suddenly decide to release with little notice and whether we should support that or not. Leaving this aside I think Continuous Delivery is a positive way of working to remove unexpected delays to releases.Any process that encourages collaboration and stress-free releases should be given strong consideration in my book.

A name is always just a name but in the case of improving processes and making sure all roles are covered I think it becomes pretty important. We need to hear everyone’s voice as we attempt to fix software development.The concepts being discussed are important and will affect us all. I personally feel that neither the ‘DevOps’ or ‘Continuous Delivery’ terms are quite right for describing an attempt to engage with all stakeholders and release timely, and functioning software to end-users. Regardless of this I believe everyone involved in software design, development and releases should be involved in deciding the best way to fix the software industry.

What do you think? Are you a Continuous Delivery or DevOps fan?

What is the Business Value of Continuous Delivery?

Michael Bolton sparked a fascinating discussion on Twitter by asking what exactly is the business value of Continuous Delivery? I’ve attempted to capture the conversation on Storify but I also want to explore the topic a little further.

Michael quite rightly pointed out that beyond CD being technically possible and cool, it is difficult to see the business value of the approach. I agree with him that there are practically no systems that could ever need to release new features every few minutes. In fact I can’t think of a single system which would require such frequent updates.

Despite this I believe that Continuous Delivery can provide value to the business.

Let’s step back for a moment and consider what ‘Business value’ actually means. It obviously means ‘providing something of value to the business’. Usually this ‘something’ will be a new feature or maybe a bug fix. Being able to release frequently could certainly get either one of those out to the business, but CD is only one possible solution to this problem. Even more important that releasing frequently is releasing reliably.

Almost every business wants to know exactly when a feature will be useable by real users. Releases require code to have been written and tested. Deploy scripts need to be reliable. Code needs to be packaged for release. Release documents, notes, maybe even marketing materials need to be produced. It seems logical to think that with so many parts to a release, and most likely the need for multiple people or even teams to be involved, you should release as infrequently as possible to reduce impact. In reality the more infrequently you perform an action the more likely you are to make a mistake.

The more frequently you practice a series if actions the more natural they feel.

Painful systems requiring workarounds or insider knowledge are often accepted if the user only has to feel the pain once in a blue moon. Bringing the pain forwards is really the only way to motivate people to fix these issues.

Simon Stewart responded to Michael Bolton’s initial Tweet questioning the business value of CD with the benefit of cleaner code. To be successful at CD you will need to learn how to be disciplined about writing and maintaining code. Developers need to be adding automated checks, and builds will need to be manged. My experience of adopting CD showed that having a goal that the developers were excited to achieve could be be a great motivator to get people fixing test environments, taking responsibility for quality, and maintaining automated checks. Obviously there are others ways that you could achieve these benefits but it would be foolish to dismiss the impact that CD can have on a development team.

Maybe the value of CD only starts to appear when you have multiple development teams contributing to a single code base. With a single developer and a single codebase the thought of releasing multiple times a day points to poorly written and tested code being endlessly thrust upon the user. If you consider a CD approach for multiple small development teams working on a shared codebase then you can see that if you do 3 releases in a day you are not necessarily saying that each developer released in that day. In fact at Songkick our release problems increased as the development team grew. Each developer was spending around 1-3 days on building and testing new features but this alongside regular bug fixing and technical improvements meant we were exceeding our release capabilities.

We could see that creating a process that allowed developers to commit code freely and receive fast feedback would be a huge improvement. Once we had made the necessarily cultural and technical changes to our process, attitude, and environments, we realised that with just a few additional changes we could ship our code directly to production without increasing the risk beyond our comfort zone. Taking this final step was hugely motivating for the development team; we wanted to create an environment where everyone has access to the information they need, can make decisions based on their expertise, and has access to support when they want it. At Songkick we did this by giving all developers the freedom to ship code whenever they were ready.

Unfortunately it seems rare for a team to sit down and define their release process, instead they rely on some process which evolves over time to handle new tools or respond to problems. Business value can only be achieved if you have code that actually makes it to the hands of your users. Being able to release the code to production in minutes doesn’t remove the need to define, design, build, and test changes but it can help you to develop a reliable release process.

I consider CD to be a technical solution to the problem of release delays. As with all releases there is likely to be some impact to the user, consider what this means and then plan accordingly.

Look at the risks of every release and identify actions to monitor or mitigate them. CD gives you a hugely flexible release process, you have the means to release quickly and frequently, but you also have the choice over whether you use it.

Instigating change

In my last post I defined different approaches that people adopt for dealing with bad practices. Consider a bad practice to be anything which is damaging to a project or your sanity. This could be something that has an obvious negative impact such as excluding testers from early feature conversations, or maybe a more personal issue such as feeling you are unable to test effectively because you have too many interruptions during the working day.

Many people just accept that company culture is set in stone, either taking for granted that things are good, or suffering through the bad times. Maybe you spend your evenings and weekends complaining to your loved ones or suffer from the Sunday night ‘back to work dread’. I’m a huge believer in individuals recognising and driving improvements to process and projects. No matter what your level within the company be pro-active in identifying changes and do whatever you can to make things better.

How you go about actually making changes is somewhat down to your personal preference. At the highest level you need to know what the problem is, have some ideas on how it could be solved or improved, and then take steps to gain buy-in from the team.

I’m trying to convince my boyfriend that we need a dishwasher. In truth I’ve been trying to convince him for about 18 months now. We both commute so washing up is ignored until we run out of something, usually the tea spoons go first. With a mountain of dishes to get through we end up spending a whole evening cleaning up. We both agree that the problem is washing up takes up too much of our free time.

Initially when I proposed a dishwasher as a solution he was a total no-no. We would lose a cupboard in the kitchen, we would need more plumbing, environmentally it was a terrible idea, and have you seen what they do to the glasses?! I left it and then gradually drip-fed him reasons why this would be a great idea. I told him about the new dishwashers that actually use less water than washing up by hand, I emphasised how much free cupboard space we have these days, I sighed over another evening spent washing up. Basically, without him even realising it he was hearing the benefits of making the change on a regular basis. Finally the glasses, I can’t disagree with him that dishwashers really do scratch glasses so I offered a compromise, even if we wash glasses by hand it will be better than having to do everything. We have reached agreement that a dishwasher is the solution to the problem. Now we just need to work out the practical aspects of achieving the goal.

Workplace problems can be resolved with a similar approach.

Identify the problem:

Make sure you are focusing on the problem and not the solution. Your problem will never be that you aren’t following Continuous Delivery or that you don’t get invited to a particular meeting. You might find that you can identify several quite different problems. In this case choose the most problematic, or the most visibly problematic to solve first. Try to understand the reasons why the problem exists, was the situation introduced intentionally and if so, why?

Find potential solutions:

Spend some time researching how you could solve this problem. Try to identify as many possible solutions as you possibly can, you might be surprised at how a seeming crazy idea can turn out to be a brilliant solution to the problem. Aim to come away with one or two potential solutions that you can use to gain buy-in from the team. Be careful not to have a fully defined solution that could make people feel like you’re trying to dictate change.

Blog posts, videos of conference talks, meetups etc can all offer ideas for solving problems even if they don’t immediately seem to have much in common. Take care not to fixate on a particular approach just because it seems trendy right now.

Set an end goal that the whole team agrees with:

To be successful you need at least the majority of the team to buy-in to your solution. How you go about this will depend on your team as well as on your personal preference. You may prefer to present to the whole team at a regular team meeting, or you might prefer to meet with people on a one-to-one basis to recruit foes to your cause. I find it helpful to sound out my ideas on several honest, and smart people within the team before presenting to everyone. Hopefully they’ll point out any obvious flaws and help you to build a stronger case. Try to address the whole team as soon as you feel confident about your solution.

There will always be resistance but there will probably also be valid concerns about making a change, or towards your proposed solution. Engage these people in conversation and make sure you understand their reasons for resisting. Work with them to help them overcome the fear of change, particularity if the proposed change will affect the way they work.

Don’t worry about how ambitious your proposed solution may seem. If everyone has a clear idea of where you want to end up then tiny steps will lead you to the goal. Keeping moving forwards and one day you will arrive at your destination.

Persevere.

Change is difficult. Do it anyway.

The conditions you face day-to-day are one of the key components to your workplace happiness. Most people have some opportunity to improve the way they work so look critically, dream big, and be courageous in driving change.