Each year, Ministry of Testing launch a #30DaysOfTesting challenge. This year the topic is ‘Quality‘. The challenge began in March, but I was a little late getting started. So, throughout April and May, I’m going to be publishing a series of blog posts where I share my progress.

Day 11 – Watch the AMA on Quality Engineering and join the conversation on The Club
Day 20 – Look up and share a definition on Quality Engineering

The Ministry of Testing regularly host Ask Me Anything webinars which allow attendees to post any question they want. These are among some of my favorite webinar series, so I was thrilled that one of the activities for this challenge included watching one of the videos.

I chose to merge this with the Day 20 task because one of the questions was about the definition of Quality Engineering.

Quality Engineering Definition from the AMA

During the webinar, Anne-Marie provided us with definitions for Quality Assurance (QA), Quality Control (QC) and Quality Engineering (QE).

  • Quality Assurance (QA) – Ensuring that a product or feature is fit for purpose.
  • Quality Control (QC) – The process behind Quality Assurance.
  • Quality Engineering (QE) – Looking at the whole ecosystem and identify ways for the team to improve quality.

For Quality Engineering, we need to understand that everyone will have a different perception of quality.

Other Thoughts About Quality Engineering

Quality Engineering involves looking at the entire development process and finding ways to improve. Just like Quality Assurance, it is about asking questions. For example:

  • How could we write code better?
  • How could we deploy better?
  • How could add new features better?
  • How could we develop a better experience for the end-user?
  • How could we get feedback better?

It is also about teamwork and improving the teams ability to find defects that may affect the quality of the product, fix them, and try to prevent future defects. We don’t just look at individual aspects of the process, but the entire thing. The transition from one stage to another should be as seamless as possible.

Maintenance is a major point as well, but not just maintenance of the product. We need to look at the maintaining the teams ability to deliver the product, ensuring that the team is able to react to change.

Activities related to quality engineering would include analysis of quality metrics, gathering feedback, understanding and prioritizing risks, and communicating this information to the different teams. This information can be used to identify ways to improve the quality of the product and the overall process of delivering that product.

Comparing Quality Assurance and Quality Engineering

This topic really got me thinking about the difference between quality assurance and quality engineering.

Quality Assurance can be a very singular role. Its main area of focus tends to be on a single product or feature within that product. It is concerned with the present state of the product and team.

Quality Engineering, on the other hand, deals with the long term goals of the product. It looks at the process as a whole rather than a single product, and how the product will be developed in the future.

That is not to say that one is more important that the other. A lot of the information provided to help with future improvements will come from what is happening now.

If we look at shift left and shift right:

  • Shift Left – Testing earlier in the lifecycle so defects are found earlier and faster
  • Shift Right – Introducing analysis and monitoring post-production to gather more information that will improve the Shift-Left approach to testing.

Quality Assurance is Shift Left, and Quality Engineering is Shift Right.

Other Ideas From the AMA

Anne-Marie mentioned the idea of small experiments. This is something she talks about in her article ‘Quality is a journey but do we know the destination‘ (which I talked about on Day 2). Identify what we want to learn and gather information by running a series of small experiments. Try things out, and see what happens. Use the outcomes of these experiments to show that we are improving quality. But remember, change won’t happen overnight. It will take time.

During the AMA, Anne-Marie was wearing an awesome T-Shirt which someone decided to ask a question about. The t-shirt shows a pyramid with criteria that metrics must meet.

Metrics must have outcomes, should have a purpose, provide regular feedback, have clear ownership and be well informed.

Some other points about metrics that were made include:

  • Avoid metrics that isolate testing to testers. Quality is about everyone, not just testers.
  • Make sure that metrics aren’t used to confirm bias. Ensure that you are actually learning something from them.
  • Identify what you actually want to achieve by using these metrics.

Further Reading

Ask Me Anything, Quality Engineering – Link to watch the webinar, free Club membership required
Metrics, T-Shirts and Quality Engineering – An article by Anne-Marie Charrett about the Metrics T-Shirt she was wearing during the webinar
Quality is a journey but do we know the destination – Another article by Anne-Marie Charrett
Ask Me Anything, Quality Engineering – further discussions and questions on The Club
30 Days of Quality Day 11: Watch Quality Engineering AMA – Day 11 of 30 Days of Quality challenge discussion on The Club
The Importance of Quality Engineering – an interesting article by Hemanth Yamjala that provided some interesting ideas for my definition of Quality Engineering
The Evolution From Testing to Quality Engineering – another interesting article published by Sam Solutions
Shift Left and Shift Right – A blog post I published about shift left and shift right


#30DaysOfTesting – Progress so far

  • Day 1 – Lookup some definitions of what ‘Quality’ is and share your own definition on The Club
  • Day 2 – Read and share a blog post on Quality
  • Day 3 – Pick a book to read that discusses ‘Quality’ and share on The Club why you’ve chosen it
    • Will be completed along with Day 30
  • Day 4 – Capture five different perspectives on Quality and share their similarities and differences.
  • Day 5 – Get some members of your team to join you in a game of Quality Jenga
    • Can’t complete due to COVID-19 Pandemic. Not sure how to play Jenga when everyone is working from home.
  • Day 6 – Find out what ‘Quality’ means to your teammates
  • Day 7 – Follow 5 people on social media who are sharing or working around ‘Quality’
  • Day 8 – Sit with your team and find out how you’re building ‘Quality’ into their work
  • Day 9 – Contribute to a discussion on The Club about ‘Quality’
  • Day 10 – Find and read feedback from your customers. What does this tell you about the quality of your product?
  • Day 11 – Watch the AMA on Quality Engineering and join the conversation on The Club
  • Day 12 – Create a visualization that can help describe the different aspects of ‘Quality’
  • Day 13 – Find, listen and share a podcast on ‘Quality’
  • Day 14 – Read about ‘Quality Characteristics’ and share a characteristic that matters to you
  • Day 15 – Ask five different teams within your organisation what is the most important quality characteristic for them
  • Day 16 – Pick a ‘Quality Characteristic’ and use it to guide an exploratory testing session
  • Day 17 – Pick an app that you use daily, what quality aspects of the app encourage you to use it?
  • Day 18 – Share a bug you have found in your system and the quality characteristic that is was potentially affecting
  • Day 19 – Read and share your thoughts on Principle 5 of Modern Testing
  • Day 20 – Look up and share a definition on Quality Engineering
  • Day 21 – Share a resource on how you might measure quality
  • Day 22 – Find out what metrics your internal stakeholders care about and why
  • Day 23 – Map out how ideas get to production and look for the biggest bottlenecks or problems
  • Day 24 – Share what quality metrics you track or what metrics you want to track
  • Day 25 – Research how an external company improves their ‘Quality’ – how can your team adopt these improvements?
  • Day 26 – Create a way to report to your stakeholders your team’s views on quality
  • Day 27 – What is the role of a quality coach?
  • Day 28 – Research the different between Functional Quality and Structural Quality
  • Day 29 – Contribute to a discussion on The Club about whether there is a difference between code quality and software quality
  • Day 30 – Share something you’ve learnt from the book you chose to read on Day 3.