The latest Ministry of Testing power hour was on Test Automation, with questions answered by Angie Jones.
At the time, I was in the process of writing out my latest blog post. This was a response to a question I was asked while giving a talk on Record and Playback in test automation at the #MidsTest meetup in Birmingham.
Since I am due to give this talk again, I decided this was a great opportunity to find out what Angie’s thoughts on the subject were. I mentioned how and why I like to use Record and Playback. Crucially, I mentioned that I adapt the code to make them more maintainable and robust.
I really liked Angie’s answer. She talked about how she discourages the use of Record and Playback, but was glad that I modified the code. This supports the message I am trying to convey in my talk. It is a matter of choice if you decide to use Record and Playback. There are many benefits, but only if it it used right. Any code generated from Record and Playback must be adapted and modified.
Angie’s thoughts on the use of Record and Playback in test automation. Answered in the Ministry of Testing Power Hour on test automation.
Open Letter to Codeless Automation Tool Vendors – Angie Jones A link shared by Angie during the power hour event. I’ve been curious about new codeless automation tools and had meaning to ask a question about this. However, I forgot to post it before the power hour. I was glad that she addressed this in her answer to my question. Her open letter addresses the issues with Record and Playback tools, and provides a list of features that a codeless automation tool should include.
After watching this talk, I made my first attempt at sketch noting. This was not done live, but while reviewing and writing up my notes after watching the talk. This was inspired by sketch notes of talks given at Collaborate Bristol conference I attended a couple of months ago. I was struck by how beautiful these sketch notes were. Unfortunately, my attempt was a little more messy. However, it gave me an opportunity to review my notes and take the most important points. It is nice to have all these main points displayed on a single page of A4 paper. This will help jog my memory when revisiting these notes in the future. I am definitely going to give this another go in the future. Next conference will be SwanseaCon on 9th September.
My first (messy) attempt at sketch noting.
Other blogs that share lists of test related articles
The Club https://club.ministryoftesting.com/ A forum for discussing, asking questions, answering questions, and requesting help. Run by the Ministry of Testing.
A few months ago I came across the term ‘Abuse Cases’ in a blog post by Nicola Owen. It was a term that I’d never come across before. This inspired me to write a blog post where I provided my own interpretation of Abuse Cases.
When Ministry of Testing announced their latest Power Hour event on Personas, I was eager to submit a question about Abuse Cases. Gem Hill beat me to it (although she didn’t use the term Abuse Cases). I see Abuse Cases as being examples of ways the application could be misused. This question didn’t just focus on ways the product could be misused, but also how a product could be attacked.
Check out The Club for a full list of questions and answers from the Power Hour.
Events
Personas – Power Hour Cassandra H. Leung dedicated an entire hour to answering questions about Personas on The Club. Questions were asked about personas templates, edge cases and personas for those who would misuse an application.
Social Media Discussion
Superhuman discussion (Twitter) Twitter discussion shared by Cassandra via the Persona Power Hour as an example of a persona created to show how an application could be misused.
Smoke Testing vs Sanity Testing After finding the discussion about smoke and sanity testing on LinkedIn, I decided to setup another discussion on The Club to see if anyone else had some ideas to share.
Testers, Please speak to the developers This week I published my write up of the 99 second talk I gave at the Birmingham test meetup last week. In this post I talk about the importance of speaking to the developers. Communication ensures that everything understands the requirements and identify ways to make the application more testable.
Other blogs that share lists of test related articles
The Club https://club.ministryoftesting.com/ A forum for discussing, asking questions, answering questions, and requesting help. Run by the Ministry of Testing.
This week I attended the Birmingham #MidsTest meetup, where I presented my talk “The Joy of Record and Playback in Test Automation” for the first time. I am due to give this same talk at TestBash Manchester and Test Con Europe, both in October this year.
I was great to be able to give this talk before these conferences. I can practice and read through each slide as much as possible, but it is nothing like presenting in front of actual people. I was able to highlight a few areas where things did and didn’t work. This allows me to make some improvements before I next give this talk.
I also gave another 99 second talks – I love these. This time I talked about the importance of communicating with developers to help with testing efforts. I’ll write this up into a blog post at some point next week.
Personas – Power Hour Ministry of testing are hosting another power hour on The Club, this time about Personas. If you have any questions, post them by 7pm on 24th July (British Summer Time).
Social Media Discussions
Smoke Testing vs Sanity Testing I was asked my opinion on a LinkedIn discussion which I was more than happy to share. It was regarding the use of smoke and sanity testing. Smoke testing is where we test the key features in an application, but only do basic checks. Sanity testing is where only a specific feature is tested, but we go a lot deeper into checking that the feature works exactly as expected. I prefer to do sanity testing when a change has been made to the application, and smoke testing every 2 – 4 weeks depending on how much the system has changed. What are your thoughts? Thanks to Bharath Selvam for bringing this post to my attention.
Excuses, Blame and Fear I started a discussion on The Club a couple of months ago on testing excuses and if they are evidence of testing incompetence, negative workplace culture, or something else. A few people have added some interesting responses to the discussion in the last week.
Difference Between Static and Dynamic Testing? Never be afraid to ask a question. The Club is a great place to ask for help and clarify someone you fully know or understand. Here, someone asks a question of the different between static and dynamic testing. Do you know?
Testability – Power Hour This event took place the week before last, and had some interesting responses. Testability is a subject that has really interested me in the last few months. Questions I posts included one on testability for manual and automated testing, what we should ask for when asking for improvements on Testability, and what to do when something cannot be controlled or observed via the UI.
Articles and Blog posts
Maybe You Don’t Need a Date Picker – Adrian Roselli Picking a date can be a complex and tedious process, especially on smaller mobile devices. Surely having to scroll through seemingly endless lists isn’t the best way?
The Club https://club.ministryoftesting.com/ A forum for discussing, asking questions, answering questions, and requesting help. Run by the Ministry of Testing.
I was recently given the opportunity to attend one of the London Tester Gathering Workshops. These took place over 3 days and included a variety of full and half day workshops – between the 26th and 28th June 2019. I’d been lucky enough to have won a ticket in a LinkedIn competition. We were asked to choose the workshop we’d like to attend and give a reason why. When I saw that one of the workshops was about learning to ‘Automate BDD scenarios using SpecFlow’, I knew I had to enter the competition. My reason was simple, at work we were hoping to start using SpecFlow to improve our current test coverage.
I learnt a great deal attending this workshop. I was sat at a table with a develop and tester from the same company who had travelled all the way from Ulverston in Cumbria (a beautiful part of the country, I have family who live up there so I travel there often). Chatting to them about the exercises really helped enhance my learning.
There was also a definite advantage of being taught about SpecFlow from the creator himself – Gáspár Nagy. He went into great detail about Behaviour Driven Development (BDD), the understanding is essential to using SpecFlow. He explained the exercises fully so that we could complete them in the workshop, or complete them at home.
This was the first time I’d attended a workshop like this one. I’m so grateful to have had the opportunity. I definitely plan to attend more in the future. The opportunity to learn about the subject from an expert (especially the creator), and discuss with other attendees provided a really positive learning experience. We were provided with enough information to apply our new skills to our own testing projects, and given enough additional resources to expand our skills further.
Lee Marshall also attended the London Tester Gathering Workshops. He attended the EventStorming; Deliberate, Collaborative Learning Between Multiple Disciplines workshop by João Rosa and Kenny Baas-Schwegler. He wrote about his experiences on his own blog: https://thepiratetester.wordpress.com/2019/07/04/2019-07-04-eventstorming-at-ltg/
Podcasts
259 – New way to learn test automation with Angie Jones Angie Jones talks about her role as a Developer Advocate at Applitools and the test automation university – a free, online platform that allows people to develop test automation skills from all over the world.
5 tips how software testers can collaborate with software developers – T. J. Maher – medium ” Developers aren’t opponents. We are teammates. We’re partners.” A brilliant article about the importance of maintaining positive relationships between developers and testers. Key ideas include encourage both to participate in planning sessions and not letting your frustrations get to you when your work is criticised.
Why the three part user story template works so well – Mike Cohn – Mountain Goat Software This article looks at the positives of the user story, that it provides context by informing us who it applies to, what they want and why they want it. It also looks at the negatives when user stories start with ‘as a user’. This can indicate lazy thinking and that the designers are unaware who the ‘users’ are.
You do not need to make the wrong assumption about your users anymore – Joel Montvelisky – qablog If a tree falls in a forest and no one is around to hear it, does it make a sound? I previously used this philosophical riddle in a lightning talk at the Fall 2019 OnlineTestConf. I looked at how the impact on bugs can dramatically increase when a bug is not ‘heard’. In this article, Joel Montvelisky provides an alternative take on this idea. If there is no sound, does this imply that the bug is not important? Should we be the ones making this assumption?
Other blogs that share lists of test related articles
The Club https://club.ministryoftesting.com/ A forum for discussing, asking questions, answering questions, and requesting help. Run by the Ministry of Testing.
Last week, I attended a conference about UX and Design. Why should a software tester attend a conference aimed at designers?
A feature may meet the technical requirements, but if it is poorly designed then the negative user experience may encourage users to stop using the application. If there are no users, then the application is pointless. User experience is an essential part of software testing. As testers, we have to make sure that the application both works and provides the user with a positive user experience. Designers are the best people to talk to about understanding and enhancing the user experience.
Monzo is an interesting banking app designed with user experience in mind. Its aim was to make banking easier by erasing the frustrations commonly associated with traditional banking. Juliana Martinhago, product designer at Monzo, delivered an interesting talk about how this was achieved. Teams were formed around outcomes not features, and getting feedback from users so that the right solution was released.
Jon Fisher, head of UX at Nomensa, talked about the possibility of poorly designed applications killing someone. A few real life examples were given to demonstrate why it is important to look at potential pain points as well as value when designing something.
Gavin Strange, director and designer at Aardman Animations (the company that produced Wallace and Gromit, Shaun the Sheep and Chicken Run) provided an energetic talk about the importance of creativity, trying out new ideas and sharing them with the world.
Catalina Butnaru discussed the principles of ethical AI and the Minimum Ethical Product approach to create ethical AI applications.
Eriol Fox, senior product designer at Ushahidi, delivered a talk on diversity – an essential topic in any design discussion. Examples of poorly designed applications that succeeded in alienating certain demographics who were written off as edge cases. Do we really want to be abandoning potential users of our products?
Applications should be designed with the end user in mind. Collaboration is the key to achieving this. Collaboration with potential users, collaboration with designers, collaboration with all stakeholders involved in the development of the application. Software testers need to ensure that the application doesn’t just work on a technical level, but also need to ensure that it works for the customer.
What has happened in the last 2 weeks? Quite a lot…
Collaborate Bristol 2019 This was the 2nd time I attended Collaborate Conf, a conference dedicated to UX and design. I wrote a series of blog posts where I summarised my own interpretation of each talk:
London Tester Gathering Workshops – Automate Scenarios with Specflow Last week I was lucky enough to attend the London Tester Gathering Workshops. There was a choice of 3 full day workshops, and 12 half day workshops which took place over 3 days. I attended the full day workshop on Specflow, run by Gaspar Nagy the creator of Specflow. This talk provided a brief introduction to Behaviour Driven Development and writing scenarios. We were then shown how to use Specflow to automate these scenarios. I’m hoping to use Specflow in my current test project. I am really excited to see how my testing strategy develops when I introduce what I learnt in this workshop.
#Midstest 99 second talk After the main talk, we were given the opportunity to deliver a 99 second talk. This time I came prepared and brought along a block from a patchwork quilt I’m currently making. Using this, I explained that a quilt is made up of several blocks. Each one has to be tested at various stages of the quilts development. If we find a defect too late, it can be very costly to fix. The same principle can be applied to testing. We shouldn’t just test the application at the end, we should run tests at all stages of the software development life-cycle. Bugs found later on cost more to fix. I published a blog post that explains this analogy in more detail – A stitch in time reduces critical defects.
Other blogs that share lists of test related articles
The Club https://club.ministryoftesting.com/ A forum for discussing, asking questions, answering questions, and requesting help. Run by the Ministry of Testing.
It is a little last minute, but I am so excited to be given the opportunity to attend the London Tester Gathering Workshops. I will be attending the ‘Automate Scenarios with SpecFlow’ workshop on the 26th June, run by Gáspár Nagy.
Specflow is a framework that uses Behaviour Driven Development. We’ve been looking into trying out new test automation frameworks at work and this is one that we’re hoping to use more extensively.
Response to ‘How do you solve a problem like Selenium?’
Last week, I wrote a little more about the AB Testing podcasts thoughts on the industry obsession with UI testing and Selenium. João Farias, who runs the thatsabug.com blog, shared a couple of interesting articles on the subject.
What do you mean by UI tests? by Mark Winteringham, Automation in Testing Are we testing the UI or testing through the UI? In this post, Mark discusses the definition of UI tests a little more. Not everything needs to be tested through the UI, but ultimately it is about risk. The risk defines the approach, not the tool.
Testing Ember Applications: First Steps by João Farias, That’s a bug João talks about his previous experience testing using Ember which tests the front end code, rather than the UI (which Selenium does).
Link to the comments can be found here. Thanks for sharing João, much appreciated!
Webinars
Ask Me Anything – Shift Left, Shift Right – Marcus Merrell A brilliant AMA, hosted by the MInistry of Testing. Many brilliant questions were asked about Shift Left and Shift Right, and Marcus Merrell gave some brilliant answers to them all.
Beware the inexperienced automation team, Beaufort Fairmont We wouldn’t allow a team to only consist of entry-level developers. So why would we allow inexperienced test automation engineers or testers with no coding experience develop automated tests?
Misc.
State of Testing Survey 2019, Practitest Practitest have published their annual state of testing survey. Have a look to see how your experiences compare with other software testers around the world.
“To those who say that “if you need testing at the end, you’re doing it wrong”, would you prefer a Boeing, or are you going Air Icarus?”—Michael Bolton
“The most exciting phrase to hear in science, the one that heralds discoveries, is not ‘Eureka!’ but ‘Now that’s funny…’”— Isaac Asimov
“Just because you’ve counted all the trees doesn’t mean you’ve seen the forest.”— Anonymous
“If we want to be serious about quality, it is time to get tired of finding bugs and start preventing their happening in the first place.”— Alan Page
“Where is the ‘any’ key?”—Homer Simpson
Other blogs that share lists of test related articles
The Club https://club.ministryoftesting.com/ A forum for discussing, asking questions, answering questions, and requesting help. Run by the Ministry of Testing.
I currently have a 40 minute commute each morning. Generally, it is an enjoyable drive with only one point where I am slowed down by rush hour traffic. Each morning, on the way to work, I listen to a podcast episode. There are several I follow, including some about software testing which are listed in my weekly ‘What I read last week’ blog post.
Last week, I listened to the AB Testing podcast – presented by Alan and Brent. This is only the 2nd time I’ve listened to this podcast. The main reason I don’t listen to it often is because the length of an episode is longer than the time it takes for me to drive to work – and I prefer to listen to an episode in one sitting. I started listening to episode 102.
One of the presenters had previously expressed his dislike for Selenium and as a result, twitter went crazy. It would seem that there are a lot of people in the testing community who are fiercely protective of Selenium. (I confess, I’ve not seen the original discussion. If anyone can send me a link to include, I would be grateful).
In my experience, there are usually 2 reasons why someone may criticize a test automation framework:
Lack of experience actually using the framework so being unable to fully utilize all its features.
Wanting to promote a different framework
It would appear that their reasons for disliking Selenium are neither of these. In episode 102 of the AB testing podcast, Alan and Brent expanded on their views a little more. They put forward some interesting points about the industry infatuation with UI testing. One point they made was that UI testing should be a last resort when the software is untestable. They also suggest that the industry obsession with UI testing is a result of the separation of developers and testers. There is no incentive for the developers to make the code testable in the first place, meaning that there is no choice but to resort to UI testing.
Alan and Brent appear to be more against UI testing in general rather than Selenium. However, Selenium has made it easier for novices to write automated tests without fully understanding how they work.
Personally, I wouldn’t be so quick to completely dismiss UI testing. However I do admit to being guilty of relying on it too much. I am aware of this issue and am hoping to work more on integration and unit testing in the future. With regards to UI testing, I believe that it is okay to use tools like Selenium so long as you fully understand what is going on in the code. Observe the code, break it down, understand it, and made any changes necessary. It should be understandable and maintainable.
Write better bug resolutions, Alexey Sotskov, Medium Who has closed off a bug with a single comment like ‘verified’? What does this mean? In this article, Alexey discusses why more detail is needed and what additional information should be included.
The testers role in agile, Ana Leyva, Abstracta In this article, Ana examines the importance of the testing role and the overall benefits of testing. Things are changing, but the tester role will always remain relevant.
The Club https://club.ministryoftesting.com/ A forum for discussing, asking questions, answering questions, and requesting help. Run by the Ministry of Testing.
