Category Archives: General

Testing Culture – Excuses, Blame and Fear

I came across an interesting article which discussed common excuses that testers make:
http://thethinkingtester.blogspot.com/2019/05/seven-excuses-software-testers-need-to.html

There were 3 which I found particularly worrying:

  • The other tester on my team missed the bug
  • If I log the bug I found in production, I’ll be asked why I didn’t find it sooner.
  • There wasn’t enough time to test

If a tester needs to use these excuses, then the issue might not be with testing but with the company culture.

A company culture where colleagues are encouraged to make excuses, attempt to shift the blame on someone else, or instil enough fear so someone is afraid to report something critical is not a good one.

“There wasn’t enough time to test”

This particular excuse will always exist and it is a valid one. There will be strict release deadlines which can’t always be pushed back. When the software has to be released, it has to be released. The tester has very little control over this. However, it doesn’t mean that the tester is entirely blameless if a critical bug makes its way to live.

It is the testers responsibility to ensure that the tests are optimised and prioritised. Tests covering the most essential features should be run first. Tests should also be optimised so that they are run as quickly and efficiently as possible.

Occasionally, the tester may be in the unfortunate situation where there isn’t enough time to test even the most essential features. It is also the testers responsibility to communicate what the overall test coverage it to those making the decision to release. They need to be aware what was tested and what wasn’t tested. This allows them to make an informed decision about if the software can be released or not.

Provided the tester prioritised and optimised their tests, and accurately communicated what was and was not tested, they can hold their head high. They tested everything to the best of their ability with the resources they were given.

“The other tester on my team missed the bug.”

With the increasing complexity of software applications, it is likely that not all members of the team will have been involved in testing every single aspect. However, is it really worth blaming the sole tester who was testing that particular feature?

Instead of blaming a single tester, we should look at the overall testing process. If there was only 1 tester validating this particular feature, maybe assigning additional testers to each feature can prevent bugs slipping through the net.

We may also want to look at how the feature was tested. If the tests were conducted using manual or automated test cases, was this particular scenario covered? If not, then the test case should be reviewed and the test coverage assessed. It may even be worth including some exploratory testing so additional unscripted scenarios are covered.

It should be recognised that we are part of a wider team and we should all be working together to improve the quality of software applications. When a bug makes it through the cracks we must remember that testers are human. They are bound to miss things out. It is not possible to investigate every possible decision that could be made when using the software.

“If I log the bug I found in production, I’ll be asked why I didn’t find it sooner.”

First of all, the person blaming the tester needs to realise that it is better that the bug was found sooner rather than later. It will be much worse if a customer finds and reports the bug.

Second, when a defect is found in production, it still needs to be fixed as soon as possible. Reducing the impact of that bug for the customer should be top priority. When a fix has been released, we can start investigating what went wrong. However, the purpose of the investigation should not be to find out who to blame but to prevent a similar situation happening again.

Finally, if there are employees terrified enough not to report critical issues, then what else are they holding back on? Employees will work much better if they are not afraid of management. Honesty should be encouraged so any mistakes can be rectified sooner rather than later. Otherwise, it will be worse for the business.

I will repeat what I said earlier, testers are human. A few bugs are likely to make it through the cracks. Software is becoming increasingly complex, there are just too many ways for the software to go wrong.

Final point…

There is a TV series that is shown in various countries – The Apprentice. It involved a several candidates who are applying for a highly paid job or money to invest in a business. The candidates are split into teams and are given a business related task. The team that makes the most money from that task wins the round, and 1 person from the other team is fired. The firing process involves the candidates being interrogated in a board room. The candidates are encouraged to blame others for the failure, and defend their own actions. One person is eventually fired.

Personally, I hate this show. It promotes a toxic culture of fear and blame. What does this actually achieve? Isn’t it better to focus on learning from the mistakes, preventing them from being made in the future and improving the process?

If the tester is forced to use any of these excuses, then it might indicate an issue with the overall company culture, not the tester.

Main image taken from publicdomainpictures.net

Advertisements

Abuse Cases – Understanding Motives

Before identifying how a user might misuse an application, we need to understand why a user might misuse the application.

“Sometimes when I try to understand a person’s motives I play a little game. I assume the worst. What’s the worst reason they could possibly have for saying what they say and doing what they do?”

Petyr Baelish, Game of Thrones Season 7 Episode 7, The Dragon and the Wolf

I was not able to attend Test Bash Brighton this year, however I do enjoy reading about other people’s experiences. Today, I came across a blog post by Nicola Owen containing a write up of her notes. Included in this post were the notes for the talk by Claire Reckless and Jahmel Harris, titled ‘United by Security: The Test that Divides Us’. Nicola included a photo of the following slide:


Slide from Test Bash Brighton talk ‘United by Security: The Test that Divides Us’ by Claire Reckless and Jahmel Harris. Photo taken by Nicola Owen.

Abuse Cases vs. Use Cases

I’m already familiar with the concept of use cases, where a specific scenario is created that suggests a way in the application may be used. The use case would demonstrate a method in which the user may interact with a product. It would then go on to demonstrate how the user would gain value from this interaction.

Abuse case would have a similar concept. Except the abuse case would suggest a situation that would be undesirable for the business. This is a specific scenario that suggests a way the application may be used, but in a way that would benefit a malicious user. What reasons might a user have for interacting with the application in an unintended way? How might they do this? Would they be successful if they tried?

One thing I really like about the idea of Abuse Cases is that they don’t just look at how a user might misuse the application. They also think about why a user might misuse the application.

Creating Abuse Cases

I decided to have a think about how a user might attempt to abuse the software applications I’m currently working on. I test applications designed to control scientific instruments and analyse data collected from running measurements with these instruments. We often sell to the pharmaceutical industry who have strict rules about how experiments are conducted and the data used. The settings used for an measurements must be logged, and the data cannot be manipulated or deleted.

A user who works for a pharmaceutical company may be under pressure to get a drug approved by the FDA. They may choose to manipulate result data, or change the way a measurement is run so that the results better support the product for FDA approval. They may also attempt to generate fake data so that the FDA can approve the drug sooner.

By understanding the motivation of a malicious user, we were then able to identify potential ways a user might attempt to misuse the application. We then investigated if it was possible for the user to do this.

Unintended Consequences

“Your scientists were so preoccupied with whether or not they could that they didn’t stop to think if they should.”

Ian Malcolm, Jurassic Park

During her keynote at the UKSTAR testing conference, Fiona Charles constantly asked the question ‘What could go wrong?’

In this talk, there were many examples of the unintended consequences of technology. Some were caused by a human which directly affected other humans. In these examples, the human will have deliberately misused the application on purpose.

There were others that had a negative effect on society and the environment. These would not be caused directly by other humans, and wouldn’t usually have been done on purpose. Examples included self service checkouts. To make it easier for a customer to quickly scan items at the checkout, there has been an increase in the amount of plastic packaging used. This has had a negative impact on the environment.

Abuse cases like these usually don’t carry a motive. This makes them a lot harder to understand or anticipate. Despite this, they are still something to think about. The best method would be to question everything – assumptions, biases, objectives and decisions.

Summary

Unlike Use Cases, which need to be successful, an Abuse Case must be unsuccessful.
When people talk about the importance of understanding the end-user, this should apply to both good and bad users. We should also think about the ethical implications of the application and the potential impact it might have on society or the environment.

By anticipating abuse cases in advance, we can create software that is safer and more secure.

Main image taken from publicdomainpictures.net

The Risk of Forgotten Knowledge

What is the most important thing in your possession right now? What would the implications be if you were to lose it?

Yesterday, I took a flight to Colorado. This is the first time I’ve been to the USA since 2008, and the first time I’ve travelled abroad for work. I am a little nervous, which doesn’t help the fact I am naturally a paranoid traveller. I am the sort of person who checks every minute that I have not lost anything. I will panic if I put my passport in the wrong pocket of my coat or bag and can’t find it later.

While waiting at the airport for the shuttle bus, we noticed a discarded pair of glasses. We all wear glasses so understand how essential they are. This then led to a discussion of what is most important, and what item we’d be most devastated about losing.

I’ve purposefully not taken anything sentimental with me on this trip so I don’t have to worry about losing these. Glasses are an obvious answer, but I have brought a spare pair so it wouldn’t be the end of the world if I lost these. I am not overly attached to my phone either, phones can be replaced and any photos on it have been backed up. Losing my passport would be problematic, but arrangements can be made to get me home safely at the end of my trip. There are items in my possession which losing would bring me a great deal of hardship. In most cases this could be fixed, not always easily but things will get better.

There is something I would be devastated to lose and could never be replaced with any kind of money. My notebooks, one of which includes notes from the UKSTAR conference I attended last week. I still haven’t written up or analysed all my notes from the talks. This is knowledge that is currently only stored in two places, my notebook and in my memory. Memories fade. This has already started as it has been a week now.

Knowledge is not just information, it is a representation of our own personal experiences and interpretations of that information. It will differ from person to person, but each person will develop new and different ideas. New ideas develop into new knowledge.

Knowledge is the most important possession we have and must be shared for two reasons. So that others can learn and develop new ideas from it and so that it is not lost and forgotten, even if the original source has not been preserved.

While on my trip I will be continuing my write ups of my UKSTAR conference notes which will be shared in a series of blog posts. I’ve also completed several tasks on the 30 days of testing challenge but have not yet completed the write ups. The next ‘what I read last week’ post will be published on Sunday. I didn’t do must reading last week because of the conference and preparing for my trip to Colorado.

Main image from http://www.publicdomainpictures.net

Communicating, speaking and debating

It is nearly the end of the year and I’ve been looking back at how I’ve changed, what I’ve learnt and what I’ve achieved. I think this year has been the most important one yet for my career.

This is the year that I started talking more. I began posting on LinkedIn about software testing, I started writing blog posts and articles for external sites, and I created my own personal blog (although I haven’t written many blog posts here yet).

This all started when I gave a lightning talk at the Spring 2018 OnlineTestConf in June on Women in Testing. My submission was very last minute as I was a little apprehensive about giving a talk to so many people, even if it was only 5 minutes. Fortunately I decided to give it a go. I enjoyed the experience so much that it prompted me to apply to more conferences. One conference submission got accepted for the Fall 2018 OnlineTestConf. I gave a 45 minute talk on automated testing, and another lightning talk.

The confidence I gained from giving these talks has helped improve as a software tester, and boosted my confidence. I started posting on LinkedIn more, and commenting on other peoples posts. At work, we were asked to write articles for the company blog in order to promote a product launch. I eagerly volunteered, something I probably wouldn’t have had the courage to do previously.

I am always grateful when people respond to my posts. I don’t care if the respondent shares my point of view. The very fact that they have taken the time to respond means that they have taken the time to read what I have to say. If they agree with me, then the acknowledgment of my opinion gives me the confidence to share my ideas more. If they disagree with me, then I am prompted to rethink my opinions. From this my ideas develop further, and sometimes change completely.

The real benefit of this increase in online activity is that I feel more able to speak out more and my skills as a software tester have improved. I have developed both professionally and personally.

A new blog – more ideas to come…

So I have finally done it, I have created my own blog. It is still a working progress but rest assured, I will add to it over time. 

I have previously written blog articles for other sites. A few for Malvern Panalytical, my current place of work, and another for testproject.io. I’ve included all my external publications on a separate page in this blog. I hope to write my own articles soon which will be published on this blog. 

In a few days, I will be giving my first ever conference talk. It is at Online Test Conf at 4:15pm (UK time) on Tuesday 27th November (the times listed on the website are either Central European time or Eastern US time). My only previous experience is a lightning talk which I gave at the same conference earlier this year. It was an amazing experience and I am so excited (and a little terrified) at being given another opportunity to give another talk. 

There are two things I really like about the Online Test Conf. One, the fact that the conference is online and free means that we can reach out to a wider audience. Second, the organizers create a slack channel for each talk that takes place, including the lightning talks. This gives everyone a chance to discuss the talk and ask additional questions once the talk is complete. I feel this gives the conference a higher level of interaction that doesn’t exist at other conferences. Usually there will be about 10 minutes for Q&A and then that’s it. With the Online Test Conf, the discussion can last for hours after the talk. 

Giving talks at conferences is something I’ve been aspiring to do since I gave my lightning talk. I hope to have additional opportunities in the future.