I’ve set myself a challenge to attend at least 3 meetups a month, which includes 1 I’ve never attended before. Click here to read about the other meetups I’ve attended this year.

Here is my progress so far:
Meetups attended in 2020: 7/36
New meetups attended in 2020: 3/12
(Last updated 10th March 2020)

Event: Feb #MidsTest
Date: 6pm Wednesday 19th February 2020
Hosts: Lee Marshall and Ben Fellows
Location: Sainsburys Store Support Centre, Coventry
Talk: Continuous Delivery – A testers journey and lessons learned
Presenter: Ahmed Hussein and Tom McCawley
Website: https://midlandstesters.wordpress.com

Here we are for the 2nd #MidsTest meetup of the year. This time I had to travel all the way to Coventry. Travelling to these faraway meetups has become a little easier. The office where I work was damaged by flooding caused by Storm Dennis (the naming of storms does not make them any friendlier), and I now have to work from home. Good news is that my home is closer to Coventry, cutting about 45 minutes off my travel time. Bad news, I have started to suffer from cabin fever. I miss my colleagues – I’m not enjoying the isolation of working remotely. It was such a relief to speak to actual people about software testing again.

The Main Talk

Tonight’s talk was given by Ahmed Hussein and Tom McCawley who gave a talk about introducing a CI/CD pipeline. I really enjoy listening to these sorts of talks – hearing about the experiences of other organisations can be really inspirational. We shouldn’t try to copy what’s been achieved as every company is different, but I find them great for developing new ideas that could be brought back to our own workplaces.

This was a great talk for anyone who didn’t know what CI/CD was, or knew very little about it. They started with the basics of what CI/CD actually was, providing a few very useful definitions.

Continuous Integration (CI) – Code merged frequently to avoid complex merge conflicts at the end of a development cycle.

Continuous Delivery (CD) – Software is always ready to be deployed, with all changes tested

Continuous Deployment (also CD) – Code commits go through automated testing, so can be automatically released.

It is good that the speakers didn’t assume that everyone knows what CI/CD means. There seems to be a lot of assumption throughout the community that everyone knows what it means. This makes it harder for those who don’t know to ask. I first heard the term a couple of years ago while reading a blog post (no idea which one), where the writer expressed dismay at the fact that so many people didn’t know what CI/CD meant. I was embarrassed to be one of those people – fortunately a little bit of googling fixed that.

The little things that help with CI/CD

Achieving a CI/CD pipeline does not happen over night. It requires the implementation of several tools and processes. Ahmed and Tom talked us through several of these:

  • Feature Toggles
    New features are deployed to live but are hidden and don’t affect existing functionality. Once they have gone through user acceptance testing (UAT), they can be switched on or off at any time.
  • Rollback Process
    It is inevitably that eventually something will go wrong. The team needed the ability to easily rollback to a previous version of the application should a critical issue be found.
  • Monitoring, Logging and Alerts
    It is better to know about an error sooner rather than later. The system is continuously monitored so that the team can be alerted to any issues as soon as possible. Alerts are sent via slack so that the team can start investigating the issue immediately.
  • Testing
    Tests included Unit tests, Consumer Driver Contract (CDC) tests, API tests, and end to end UI tests. Exploratory testing and Performance testing was also done. There was an aim to achieve the ‘perfect’ test pyramid, but the team ended up with more UI tests. This was due to testing on Internet Explorer becoming more essential.
An ideal test pyramid showing that there needs to be fewer UI tests, and the actual test structure which had more UI tests.
  • Builds
    When creating new builds, the team have to trust that it will provide the correct feedback. Every build had to have tests, with no tests the build would fail. The daily stand-up meetups included checking that there had been no failed tests. Alerts were sent via slack so the team were aware of failed tests as soon as possible. Failed tests were investigated even if they passed when rerun.
  • Team Discipline
    • The team made sure that there was an adherence to scrum ceremonies – this included backlog refinement sessions, retrospectives, demos and daily stand-ups. These would happen even if there was nothing to show or discuss so the team wouldn’t fall into the habit of giving them a miss.
    • Each user story underwent a three amigos session, and tests were written in parallel to development. If the team didn’t know how to test it, then they wouldn’t start development either.

Did it work?

The aim was to start releasing multiple times a week – enabling urgent features to be completed within hours of the ticket being written. This was a stark difference to releasing every 2 – 3 months, so everyone had to be on board. This was quite the culture shift, fortunately there was a lot of motivation to get this working. Reducing the likelihood of failure, and awareness of the ability to roll back, resulted in an increase of confidence throughout the organisation and less hesitancy about releasing more often.

99 Second Talks

The session ended with the 99 second talk round – an open mic style session where anyone from the audience can volunteer to give a short talk. I love lightning talk style sessions, and 99 seconds give multiple people a chance to share their ideas.

My talk was about misleading statistics. It was inspired by a news article I read which stated that the number of Coronavirus cases in the UK have doubled. This sounds absolutely terrifying, until you read the article and realize that the number of cases has increased by 4 to 8. I do sympathise with those 8 people and those in countries which have reported a significantly higher number of cases. However, for those living in the UK, I don’t believe it is quite the time to panic – I hope this doesn’t change!

In contrast, I was once involved in a discussion about testing on Internet Explorer. There were some suggestions that it was all unnecessary since only 2% of customers actually use it. Now, if you have 1 million customers then a critical defect that only affects Internet Explorer could result in a reduction of 20,000 orders.

Which is greater? Double 4 or 2% of 1 million?

Relying on statistics can save time but we must remember that they can be misleading. Better decisions can be made by taking the time to investigate the full story rather than just relying on statistics alone.

Click here to see more blog posts about meetups I’ve attended in 2020.