Each year, Ministry of Testing launch a #30DaysOfTesting challenge. This year the topic is ‘Quality‘. The challenge began in March, but I was a little late getting started. So, throughout April and May, I’m going to be publishing a series of blog posts where I share my progress.

Day 12 – Create a visualization that can help describe the different aspects of ‘Quality’
Day 21 – Share a resource on how you might measure quality

On Day 1 of the 30 Days of Quality challenge, I discussed the idea that quality could be measured by listing the most important quality aspects of an application, placing these in a pyramid by order of importance and then identifying the level on the pyramid that has been reached.

This concept would work in a way that is similar to Maslows Hierarchy of Needs. Higher levels of the pyramid cannot be reached unless the lower levels have also been reached.

With this in mind, I took the eight dimensions of quality and used them to created a new version of the pyramid.

This works a little differently from my earlier idea as a high level quality can still be achieved for the upper levels of the pyramid when they are assessed independently, even if it doesn’t exist on lower levels . However, the overall quality of each level is still dependent on the lower levels. The application can look amazing, and do all the things it is meant to do. However, this would be pointless if the application does not work.

The order that each dimension appears in the pyramid will vary depending on the business priorities. The order that has been used here reflect my own opinion of what should be prioritized. Quality cannot be achieved unless the application works, and this version of the pyramid reflects this point of view. The lower levels of the pyramid represent the foundations of most applications. Without them, the product will not be stable.

Perceived quality appears at the top of the pyramid because it is directly impacted by all 7 dimensions.

Measuring Quality

For Day 21 of the challenge, I was required to share a resource on how we might measure quality. A business will choose to measure quality based on its own goals and priorities. However, while creating my own visualization of quality, I was inspired to also devise a way that quality could be measured using the eight dimensions of quality.

Each level of the pyramid is giving an independent quality score. This is a rating out of 10. For this, you can group levels together or give each quality aspect its own level. The order of the levels can vary depending on the priories of the business. In this example, I chose to separate out each level.

I used the following formula to calculate the perceived quality for each level:

W = (Y * Z)/X

W = Perceived score for current level
Y = Perceived score of lower level
Z = Independent score of the current level
X = Maximum possible score

The lowest level (performance) would be calculated first, followed by the level above it (reliability) and so on until the top of the pyramid has been reached.

Here is how an overall perceived quality score of 0.22 (or 22%) would be calculated:

Individual Quality ScorePerceived Quality Score
Aesthetics40.22
Features80.55
Conformance80.69
Durability90.86
Serviceability40.96
Reliability32.40
Performance88.00

The perceived quality score for each row is a reduction of the perceived quality score for the row beneath it. The amount the score is reduced by on each level is dependent of the individual quality score of the current level. This demonstrates how the perceived quality is directly dependent on the quality of the levels beneath it. The lower levels have a higher impact on the overall quality, because without them the rest of the pyramid will break.

The quality score can be increased by arranging the levels, if doing so is in the best interests of the business. Some business’ may place higher value on aesthetics, so may choose to have this level further down the pyramid.

If the quality of one level increases, then the quality of higher levels will improve. However, the levels beneath the level which was improved will remain unaffected. Here is what happens if the Reliability and Serviceability scores are increased to 10. It improved the score of all the levels above it, however the Performance score is unaffected.

Individual Quality Score Perceived Quality Score
Aesthetics41.84
Features84.61
Conformance85.76
Durability97.20
Serviceability108.00
Reliability108.00
Performance88.00

To increase the perceived quality score, we need to focus on improving the quality score for all levels.

Conclusion

This was a idea that I came up with while completing the 30 days of quality. It is purely theoretical, but I think that it could be developed further with time.

The idea was based on the fact that certain quality aspects are more important than others, but these levels may go unnoticed. Some of the more essential dimensions are hidden to the end-user.

The eye at the top of the pyramid is used because the perception of quality will vary depending on who is looking (some inspiration also came from the Illuminati symbol). Some people observing an application from the top of the pyramid will only see the top levels. Others will see all the way to the bottom.

Further Reading

Eight dimensions of quality – Wikipedia article
Maslow’s Hierarchy of Needs – Saul Mcleod, Simply Psychology

Earlier Ideas

Here are some sketch notes and an earlier example of the pyramid showing how this idea developed.

Initial idea for a visualization that describes the different aspects of quality. This shows how I grouped the different dimensions together by thinking what what questions they answer.
Initial sketch of the pyramid

#30DaysOfTesting – Progress so far

  • Day 1 – Lookup some definitions of what ‘Quality’ is and share your own definition on The Club
  • Day 2 – Read and share a blog post on Quality
  • Day 3 – Pick a book to read that discusses ‘Quality’ and share on The Club why you’ve chosen it
    • Will be completed along with Day 30
  • Day 4 – Capture five different perspectives on Quality and share their similarities and differences.
  • Day 5 – Get some members of your team to join you in a game of Quality Jenga
    • Can’t complete due to COVID-19 Pandemic. Not sure how to play Jenga when everyone is working from home.
  • Day 6 – Find out what ‘Quality’ means to your teammates
  • Day 7 – Follow 5 people on social media who are sharing or working around ‘Quality’
  • Day 8 – Sit with your team and find out how you’re building ‘Quality’ into their work
  • Day 9 – Contribute to a discussion on The Club about ‘Quality’
  • Day 10 – Find and read feedback from your customers. What does this tell you about the quality of your product?
  • Day 11 – Watch the AMA on Quality Engineering and join the conversation on The Club
  • Day 12 – Create a visualization that can help describe the different aspects of ‘Quality’
  • Day 13 – Find, listen and share a podcast on ‘Quality’
  • Day 14 – Read about ‘Quality Characteristics’ and share a characteristic that matters to you
  • Day 15 – Ask five different teams within your organisation what is the most important quality characteristic for them
  • Day 16 – Pick a ‘Quality Characteristic’ and use it to guide an exploratory testing session
  • Day 17 – Pick an app that you use daily, what quality aspects of the app encourage you to use it?
  • Day 18 – Share a bug you have found in your system and the quality characteristic that is was potentially affecting
  • Day 19 – Read and share your thoughts on Principle 5 of Modern Testing
  • Day 20 – Look up and share a definition on Quality Engineering
  • Day 21 – Share a resource on how you might measure quality
  • Day 22 – Find out what metrics your internal stakeholders care about and why
  • Day 23 – Map out how ideas get to production and look for the biggest bottlenecks or problems
  • Day 24 – Share what quality metrics you track or what metrics you want to track
  • Day 25 – Research how an external company improves their ‘Quality’ – how can your team adopt these improvements?
  • Day 26 – Create a way to report to your stakeholders your team’s views on quality
  • Day 27 – What is the role of a quality coach?
  • Day 28 – Research the different between Functional Quality and Structural Quality
  • Day 29 – Contribute to a discussion on The Club about whether there is a difference between code quality and software quality
  • Day 30 – Share something you’ve learnt from the book you chose to read on Day 3.