A look back at 2014

2014 was the year that I resolved to do less stuff. I have a terrible time turning things down. Empty calendars seem to fill me with fear, and I simply can’t help getting involved in things I believe in. Unsurprisingly I ended up busier than ever.

I attended, or spoke at some fantastic conferences – TestBash, Let’s Test, Agile Cambridge, BBC Develop, and EuroSTAR. At each event I met, or re-acquainted with fantastic testers from around the world. My conference highlight of year was being involved in organising Pipeline. It took months of hard work but it was hugely satisfying to see the conference unfold on the day.

A few difficult months of writing and editing finally saw my contribution included in Build Quality In. If you’re interested in Continuous Delivery this book contains some fantastic accounts of how different teams have approached it. Steve and Matthew did a great job of selecting contributors and editing the submissions.

After many years of deferring I finally completed the Black Box Software Testing Foundation certificate, and attended James Bach’s RST course in Brighton. One day of Security Testing training with Bill Matthews completed my training for the year. My greatest takeaway from all of these courses was a realisation that I do have the time, and the desire, to learn. Hopefully I’ll be building on this in 2015.

Unexpectedly I found myself co-facilitating Weekend Testing Europe. Meeting, and having the opportunity to learn with the Weekend Testing attendees has been a pleasure. Alongside Neil and Dan I’m excited about the sessions we have (almost) planned for 2015.

Podcasts seemed to be a theme of my year. Steve and Dan were kind enough to have me on Testing in the Pub twice and I wrapped up my year with a great chat with Rosie for The Minitry of Testing Dojo.

Throughout 2014 I have thoroughly enjoyed all the interactions with so many testers – thank you to all of you.

Resolutions for 2015? Well I think I might have another go at being just a little less busy…


Build Quality In

Build Quality In

I’m excited to announce that Songkick‘s experience of moving to Continuous Deployment will be included in the forthcoming book “Build Quality In” edited by Steve Smith and Matthew Skelton.

The book, available on Leanpub aims to gather together experiences, lessons learnt, and the highs and lows of building quality in. I’ll be contributing my experience of moving to continuous deployment and in particular taking a look at the impact the change had on testing and quality at Songkick.

When good practices turn bad

Over time everything evolves. Project’s evolve as new people join and as new information becomes available. Testing evolves as we learn to think and reflect on past projects. What was considered good testing ten years ago is probably not considered good testing today. There are processes and ideas that I look back on and wonder not only how I could have ever considered it to have been a good thing but to actually wonder how on earth it even worked. In my defense I should say that at the time, on that particular project, it did work.
Sometimes you’re working with something that was always known to be a bit of a compromise. We can’t be perfect all of time, for most of us we can’t be perfect for any of the time. All of this leads to the inevitable situation where you’ll be following a process, or carrying out a task and you’ll know that this is actually a bad thing* to be doing.
For example I know that test plans take far too much time to produce and maintain. I know that they can actually limit your thinking and cause you to miss bugs. However I currently work in an environment where sometimes I need people who aren’t testers to carry out some checking. I have to accept that they are only comfortable doing this if they have something to follow. Since I believe our process is effective, and I know our process depends on people besides myself (the only tester) testing, I’m reasonably OK with the need to support test planning to some degree. Obviously I can work on different approaches to this test planning but right now I’m OK to write test plans if they get me to a better place in the short-term, by improving things for the end-user.
Estimating is another common area of contention. We all know that it is impossible to estimate accurately. The anchoring effect will make sure that any number at all could be the resulting estimate. Yet, despite this, there are many people estimating work on a regular basis.
An individual taking a stand is unlikely to have any impact on a project. If you suddenly decide to stop providing estimates, then I doubt everyone else on the project will sigh with relief and drop the whole charade. It’s far more likely that you’ll be excluded from a meeting which could contain genuinely useful conversations. You’ll probably also end up being assigned estimates for your work, and then somehow expected to meet them.
So what do you do? Is it enough to go through the motions in an I KNOW THIS IS BAD kind of way similar to the way you know the late night kebab is bad but you do it anyway?Do you adopt stealth tactics and slowly try to raise a rebellion? Depending on the activity and project this might be successful. Unfortunately there will usually be at least one supporter for every activity, no matter how antiquated it has become. Maybe you’ll be satisfied that you at least had a go at making a change even if it wasn’t successful. This approach seems to be particularly common if the person supporting the bad thing is your manager.
Stealth tactics can be effective for making the small early steps. They can be used gather some evidence to support your case, or for identifying who will be open to change and who will resist. Unfortunately at some point you will have to come out into the open and launch a full-scale mission to rid your project of the bad practice. Keith Klain has given some great presentations about how to go about this. The biggest takeaway is probably that you need a lot of energy and perseverance to succeed.
So what do you do? Are you a ‘accept it as it comes’, ‘stealth changer’ or a ‘full-scale mission’ kind of person?

*Obviously there are measures of bad, I’m reasonably open-minded to things which are mostly bad but give you some overall gains. But there are also things which are just bad, bad, where the perceived gains are actually just an illusion hiding the reality of your actions.

Testing for SEO

If you ever find yourself testing a publicly available website then you should be considering Search Engine Optimisation (SEO). Basically this means ‘How easily can the website be found and indexed by search engines such as Google?’. Many users, maybe the majority of users, will go to somewhere like Google and search for a website name rather than entering the URL in the address bar. How far down the search results you appear can have a dramatic impact on the amount of traffic your website receives.
Websites with good SEO will usually have a clear site hierarchy and contain text links which provide access to all pages. Websites are indexed by bots which spider the site. Pages with no links leading to them are unlikely to be indexed simply because the bot cannot discover them. Bots do not behave like a real user so you should take care to actively consider SEO during testing.
Site structure is important for SEO, although possibly not something you can influence by testing. Do check that your site has a Robots.txt file to help Bots understand the site structure.
Search engines reward visitor satisfaction (likely a combination of visit duration and return visits), and site performance so make sure you thoroughly test the website in all supported web browsers and monitor page load times at the very least. If you have the opportunity to carry out more detailed performance monitoring or testing then take it.
When testing a website for SEO consider using the Lynx browser, a text only browser, to see the site as a bot would. If you can’t access any of the content when viewed text-only then neither will the indexing bot. All non-text content such as Flash videos or images which portray information, should have a descriptive html tag to provide the same information in a text form. Descriptive ALT tags will help your site’s accessibility as well as your SEO.
The search result link that you see in a search engine is the page title so it makes sense that page titles should be descriptive and unique. Either check the <TITLE> html tag or simply raise your eyes and read the top of your browser for every page you view. Search engines take the first terms to be the most important so it is usual to display relevant page keywords before the site name. Make sure the keywords are relevant to the page you are viewing.
Use scripts or test tools to make sure your website doesn’t contain any broken links and to check for invalid HTML.
Webpages requiring session ids or other arguments in their path are unlikely to be accessible to the bot. Depending on how your website works this may or may not be a good thing. For example I’m fairly sure you wouldn’t want users dropping in part way through a checkout flow simply because they clicked a link on Google.
A quick post-release check in Google to see exactly what has been indexed and how highly is also worth carrying out. Remember that the results you see will often be personalised to you so a website you view frequently should rank higher when you search for it.

Finally, although you want search engines to index the site, they can cause you issues with load. Use a tool such as Firebug for Firefox or the Chrome browser developer tools to check for the If-Modified-Since HTTP header, it probably won’t help in every case but in general this should allow your web servers to tell Google whether content has changed since the page was last crawled.

Benefits Me vs. Benefits the Project

New Year resolutions are usually a time to drop some bad habits, pick up some good habits, and generally try to make yourself a better human being. As well as personal goals for the year I like to set myself a quarterly work-related goal. Usually the goal focuses on solving a particular problem and consists of mini goals to keep me motivated. Previous goals have led to us adopting Continuous Delivery, tackling load testing, and introducing a bug tracking system. When I started to think about a suitable goal for the start of 2014 I realised that achieving the potential goals would have differing levels of value for me and for my current project(s). 
On my current project we use checklists and sometimes even test plans to give the product managers confidence that the developers have checked the important aspects of the feature. Yes, the developers are testing, ask me about it if you want to know more. I come along later and carry out some testing to make sure we’re not losing the plot. I have a personal goal to get rid of the test plans and to hugely simplify the checklists because I want to spend more of my time doing things that I enjoy and that I feel are of greater value. Things like exploratory testing for example. 
It would be easy to set this as my Q1 goal for 2014. I’m motivated to make some changes and I have plenty of ideas about how we could go about making these changes. But I realised that this is my goal not my project’s. The product managers and developers like the current process; they find it reassuring to have a finite list of things to check. Although I think we would be in a better place if we change this I have come to realise that you should always fix the problem that people feel is currently the biggest problem. With this in mind it looks like solving the flaky Facebook tests and finding a way to monitor CSS quality will be of far greater value to the project as a whole. 
Do you set quarterly goals? If so how do you decide on them?

Asking “Why?”

I once worked on a large project that required us to hire lots of new people. New people are fantastic. As well as having more hands to help with the work you get a fresh pair of eyes and a new set of experiences and opinions. I loved the moment when a new person would ask ‘Why?’ after hearing how some part of the process worked. “What do you mean ‘Why’?” I would ask. “Why do you do it that way?” Many a process was refined or completely changed as a result of those early conversations.
Asking ‘Why’ is nothing new. Children are famous for their never-ending ‘Why’ questions. Rather than just trying to annoy you recent studies indicate that they are actively learning when they do it .
The ‘5-Why’s’ analysis method is a popular method for understanding why problems happen. If you manage to avoid the endless loop and dig down deeper to the actual cause of the problem then you have a powerful method for identifying improvements.
As useful as the 5-Whys method is I still feel something is missing. A 5-Why’s analysis session is common when trying to understand and resolve an issue. But why are we waiting for the problems to happen before we start asking ‘Why’? New people joining projects and children learning are missing the familiarity that makes the rest of us complacent. As you follow your process, log a bug, or join a meeting about a new feature ask yourself ‘Why?’
Millions saw the apple fall, but Newton was the one who asked why. – Bernard Baruch

Choosing Continuous Delivery

Continuous Delivery (CD) is looking like it’s the new Agile. Teams hear about the wonderful things that happen with you implement CD and rush to get on-board. In reality CD is the same as any process, there are pros and cons to adopting the approach. Not every team could. or even should want to change their culture and process to allow CD to work. However if you have a team, including business people, who are willing to take the steps required then CD can have a huge impact on team morale.
Process change is hard. You are asking a group of people to change the way they do things. Not all of these people will agree that you’re moving to a better approach and so at least some resistance is normal. CD requires everyone on the team to trust each other. You need to work together to agree on a process that allows you to move fast and deploy code to production far more frequently that you are used to. You need to understand and agree on risk levels and technical changes. How will you test the builds and who gets to make the decision as to what gets deployed? Most important of all you need to agree on what will happen when things go wrong.
Despite the obvious difficulties of making these changes I still believe CD is a positive process change to make. Full team conversations around acceptable risk are always going to be a useful thing. Actually sitting down and defining a process which can then be completely managed by a tool such as Jenkins will make life easier for everyone. Most significant of all are the benefits of creating a culture where developers are trusted to release code as and when they need to.
If you would like to hear more about how Songkick moved to CD then join me for the EuroSTAR Webinar Wednesday 11th December 2pm GMT

Next time will be better

Sometimes you don’t manage to do your best testing. Even as you frantically try to test more, to test better, you know that on this occasion you’re not going to walk away feeling like you did a great job. 
This was me last week. We were facing down an exciting deadline. Time was short. I had started on the back foot by missing the earliest project conversations. From then on I felt that I was chasing but never quite catching the conversations and decisions. It didn’t help that things were changing, nothing major but just a constant state of flux around the copy, the design, and some confusion over what was going to be changed before the launch and what could wait until afterwards. 
The tight timescales meant we couldn’t follow our usual processes, developers were focused on writing code but we couldn’t test as we went. Code was ‘deployed to production but the launch would be a ‘flip of a switch’ at an agreed time. End to end testing gained an unfamiliar significance and obviously exposed a raft of bugs. 
Tight deadlines often end up with a tester co-ordinating testing meaning valuable testing time is wasted telling non-testers how to raise bugs or showing them which area of the system to test. 
Was the project a success? Yes. We hit the deadline. There were no unexpected reports from users. Everyone was happy. But I knew I could, and should, have done more. Getting bogged down by a project prevents you from thinking clearly. When you don’t think clearly you miss silly, obvious bugs. 
So what can I do? I’m going to go away and come up with a plan. A plan to guide me through these short, frantic projects. A blueprint to co-ordinate end-to-end testing with non-testers. A checklist to remind me of the silly bugs that mobile browsers can introduce, or the need to check plain copy emails. Most important of all I’m going to come up with a plan for gathering project information and finding a way to think even when imd is tight. 
Next time will be better. 

Why Every Tester Should Have a Blog

Testing is a journey. Each testing session forces you to ask new questions, review assumptions and hopefully learn something along the way. Like all journeys there will be problems, missed bugs or unexpected delays that force you to adapt your approach.

Sometimes it’s the difficult times that actually teach you how to become a better person. I am far more likely to recall the time I failed in the diplomacy stakes or remember every little detail about the critical bug that somehow made it to production. Digging in to the reason for these failures and finding something positive to take away can make mistakes a worthwhile pastime. Recording these lessons will help you apply them next time.

So much of testing relies on the tester having strong communication skills. Being able to write clearly and concisely is half the challenge when reporting bugs or reporting on testing. Writing a personal blog is the perfect place to practice your writing, as a useful aside it also helps you explore your own ideas about testing and record all those great, and not so great ideas that you have.

Many potential bloggers are deterred by the amount of time that they think a blog will require. Obviously the most successful blogs are updated frequently and I certainly find it easier to complete a post if I maintain a reasonably regular writing schedule. However it is your personal blog, no one will die if you don’t post for 6 months. Write when you have time and write when you have something to explore and enjoy the journey.

Do you have blog? If not, why not?

Book Recommendations

On Friday I asked the Twitterverse to recommend books about process change. I was hoping to find 2-3 good titles which I could give away during my Agile Cambridge Tutorial. As always the great mix of people on Twitter didn’t disappoint. Here is the full list of recommendations:

Fish, and Fish! Sticks were recommended by Erik Brickarp

Understanding Organisations was recommended by Graham Bleach

Quality Software Management: Vol 4: Anticipating Change was recommended by Michael Bolton

Peopleware: Productive Projects and Teams was recommended by Helena Jeret-Mäe

Bob Marshall linked to his fascinating blog post on Homogeneous Mindsets.

Have you read any of the recommended books? Would you recommend any others?