The latest Ministry of Testing power hour was on Test Automation, with questions answered by Angie Jones.
At the time, I was in the process of writing out my latest blog post. This was a response to a question I was asked while giving a talk on Record and Playback in test automation at the #MidsTest meetup in Birmingham.
Since I am due to give this talk again, I decided this was a great opportunity to find out what Angie’s thoughts on the subject were. I mentioned how and why I like to use Record and Playback. Crucially, I mentioned that I adapt the code to make them more maintainable and robust.
I really liked Angie’s answer. She talked about how she discourages the use of Record and Playback, but was glad that I modified the code. This supports the message I am trying to convey in my talk. It is a matter of choice if you decide to use Record and Playback. There are many benefits, but only if it it used right. Any code generated from Record and Playback must be adapted and modified.
Open Letter to Codeless Automation Tool Vendors – Angie Jones A link shared by Angie during the power hour event. I’ve been curious about new codeless automation tools and had meaning to ask a question about this. However, I forgot to post it before the power hour. I was glad that she addressed this in her answer to my question. Her open letter addresses the issues with Record and Playback tools, and provides a list of features that a codeless automation tool should include.
After watching this talk, I made my first attempt at sketch noting. This was not done live, but while reviewing and writing up my notes after watching the talk. This was inspired by sketch notes of talks given at Collaborate Bristol conference I attended a couple of months ago. I was struck by how beautiful these sketch notes were. Unfortunately, my attempt was a little more messy. However, it gave me an opportunity to review my notes and take the most important points. It is nice to have all these main points displayed on a single page of A4 paper. This will help jog my memory when revisiting these notes in the future. I am definitely going to give this another go in the future. Next conference will be SwanseaCon on 9th September.
Other blogs that share lists of test related articles
A few months ago I came across the term ‘Abuse Cases’ in a blog post by Nicola Owen. It was a term that I’d never come across before. This inspired me to write a blog post where I provided my own interpretation of Abuse Cases.
When Ministry of Testing announced their latest Power Hour event on Personas, I was eager to submit a question about Abuse Cases. Gem Hill beat me to it (although she didn’t use the term Abuse Cases). I see Abuse Cases as being examples of ways the application could be misused. This question didn’t just focus on ways the product could be misused, but also how a product could be attacked.
Check out The Club for a full list of questions and answers from the Power Hour.
Personas – Power Hour Cassandra H. Leung dedicated an entire hour to answering questions about Personas on The Club. Questions were asked about personas templates, edge cases and personas for those who would misuse an application.
Social Media Discussion
Superhuman discussion (Twitter) Twitter discussion shared by Cassandra via the Persona Power Hour as an example of a persona created to show how an application could be misused.
Smoke Testing vs Sanity Testing After finding the discussion about smoke and sanity testing on LinkedIn, I decided to setup another discussion on The Club to see if anyone else had some ideas to share.
Testers, Please speak to the developers This week I published my write up of the 99 second talk I gave at the Birmingham test meetup last week. In this post I talk about the importance of speaking to the developers. Communication ensures that everything understands the requirements and identify ways to make the application more testable.
Other blogs that share lists of test related articles
This week I attended the Birmingham #MidsTest meetup, where I presented my talk “The Joy of Record and Playback in Test Automation” for the first time. I am due to give this same talk at TestBash Manchester and Test Con Europe, both in October this year.
I was great to be able to give this talk before these conferences. I can practice and read through each slide as much as possible, but it is nothing like presenting in front of actual people. I was able to highlight a few areas where things did and didn’t work. This allows me to make some improvements before I next give this talk.
I also gave another 99 second talks – I love these. This time I talked about the importance of communicating with developers to help with testing efforts. I’ll write this up into a blog post at some point next week.
Personas – Power Hour Ministry of testing are hosting another power hour on The Club, this time about Personas. If you have any questions, post them by 7pm on 24th July (British Summer Time).
Social Media Discussions
Smoke Testing vs Sanity Testing I was asked my opinion on a LinkedIn discussion which I was more than happy to share. It was regarding the use of smoke and sanity testing. Smoke testing is where we test the key features in an application, but only do basic checks. Sanity testing is where only a specific feature is tested, but we go a lot deeper into checking that the feature works exactly as expected. I prefer to do sanity testing when a change has been made to the application, and smoke testing every 2 – 4 weeks depending on how much the system has changed. What are your thoughts? Thanks to Bharath Selvam for bringing this post to my attention.
Excuses, Blame and Fear I started a discussion on The Club a couple of months ago on testing excuses and if they are evidence of testing incompetence, negative workplace culture, or something else. A few people have added some interesting responses to the discussion in the last week.
Difference Between Static and Dynamic Testing? Never be afraid to ask a question. The Club is a great place to ask for help and clarify someone you fully know or understand. Here, someone asks a question of the different between static and dynamic testing. Do you know?
Testability – Power Hour This event took place the week before last, and had some interesting responses. Testability is a subject that has really interested me in the last few months. Questions I posts included one on testability for manual and automated testing, what we should ask for when asking for improvements on Testability, and what to do when something cannot be controlled or observed via the UI.
I was recently given the opportunity to attend one of the London Tester Gathering Workshops. These took place over 3 days and included a variety of full and half day workshops – between the 26th and 28th June 2019. I’d been lucky enough to have won a ticket in a LinkedIn competition. We were asked to choose the workshop we’d like to attend and give a reason why. When I saw that one of the workshops was about learning to ‘Automate BDD scenarios using SpecFlow’, I knew I had to enter the competition. My reason was simple, at work we were hoping to start using SpecFlow to improve our current test coverage.
I learnt a great deal attending this workshop. I was sat at a table with a develop and tester from the same company who had travelled all the way from Ulverston in Cumbria (a beautiful part of the country, I have family who live up there so I travel there often). Chatting to them about the exercises really helped enhance my learning.
There was also a definite advantage of being taught about SpecFlow from the creator himself – Gáspár Nagy. He went into great detail about Behaviour Driven Development (BDD), the understanding is essential to using SpecFlow. He explained the exercises fully so that we could complete them in the workshop, or complete them at home.
This was the first time I’d attended a workshop like this one. I’m so grateful to have had the opportunity. I definitely plan to attend more in the future. The opportunity to learn about the subject from an expert (especially the creator), and discuss with other attendees provided a really positive learning experience. We were provided with enough information to apply our new skills to our own testing projects, and given enough additional resources to expand our skills further.
Last week, I attended a conference about UX and Design. Why should a software tester attend a conference aimed at designers?
A feature may meet the technical requirements, but if it is poorly designed then the negative user experience may encourage users to stop using the application. If there are no users, then the application is pointless. User experience is an essential part of software testing. As testers, we have to make sure that the application both works and provides the user with a positive user experience. Designers are the best people to talk to about understanding and enhancing the user experience.
Monzo is an interesting banking app designed with user experience in mind. Its aim was to make banking easier by erasing the frustrations commonly associated with traditional banking. Juliana Martinhago, product designer at Monzo, delivered an interesting talk about how this was achieved. Teams were formed around outcomes not features, and getting feedback from users so that the right solution was released.
Jon Fisher, head of UX at Nomensa, talked about the possibility of poorly designed applications killing someone. A few real life examples were given to demonstrate why it is important to look at potential pain points as well as value when designing something.
Gavin Strange, director and designer at Aardman Animations (the company that produced Wallace and Gromit, Shaun the Sheep and Chicken Run) provided an energetic talk about the importance of creativity, trying out new ideas and sharing them with the world.
Catalina Butnaru discussed the principles of ethical AI and the Minimum Ethical Product approach to create ethical AI applications.
Eriol Fox, senior product designer at Ushahidi, delivered a talk on diversity – an essential topic in any design discussion. Examples of poorly designed applications that succeeded in alienating certain demographics who were written off as edge cases. Do we really want to be abandoning potential users of our products?
Applications should be designed with the end user in mind. Collaboration is the key to achieving this. Collaboration with potential users, collaboration with designers, collaboration with all stakeholders involved in the development of the application. Software testers need to ensure that the application doesn’t just work on a technical level, but also need to ensure that it works for the customer.
What has happened in the last 2 weeks? Quite a lot…
Collaborate Bristol 2019 This was the 2nd time I attended Collaborate Conf, a conference dedicated to UX and design. I wrote a series of blog posts where I summarised my own interpretation of each talk:
London Tester Gathering Workshops – Automate Scenarios with Specflow Last week I was lucky enough to attend the London Tester Gathering Workshops. There was a choice of 3 full day workshops, and 12 half day workshops which took place over 3 days. I attended the full day workshop on Specflow, run by Gaspar Nagy the creator of Specflow. This talk provided a brief introduction to Behaviour Driven Development and writing scenarios. We were then shown how to use Specflow to automate these scenarios. I’m hoping to use Specflow in my current test project. I am really excited to see how my testing strategy develops when I introduce what I learnt in this workshop.
#Midstest 99 second talk After the main talk, we were given the opportunity to deliver a 99 second talk. This time I came prepared and brought along a block from a patchwork quilt I’m currently making. Using this, I explained that a quilt is made up of several blocks. Each one has to be tested at various stages of the quilts development. If we find a defect too late, it can be very costly to fix. The same principle can be applied to testing. We shouldn’t just test the application at the end, we should run tests at all stages of the software development life-cycle. Bugs found later on cost more to fix. I published a blog post that explains this analogy in more detail – A stitch in time reduces critical defects.
Other blogs that share lists of test related articles
It is a little last minute, but I am so excited to be given the opportunity to attend the London Tester Gathering Workshops. I will be attending the ‘Automate Scenarios with SpecFlow’ workshop on the 26th June, run by Gáspár Nagy.
Specflow is a framework that uses Behaviour Driven Development. We’ve been looking into trying out new test automation frameworks at work and this is one that we’re hoping to use more extensively.
Response to ‘How do you solve a problem like Selenium?’
Last week, I wrote a little more about the AB Testing podcasts thoughts on the industry obsession with UI testing and Selenium. João Farias, who runs the thatsabug.com blog, shared a couple of interesting articles on the subject.
What do you mean by UI tests? by Mark Winteringham, Automation in Testing Are we testing the UI or testing through the UI? In this post, Mark discusses the definition of UI tests a little more. Not everything needs to be tested through the UI, but ultimately it is about risk. The risk defines the approach, not the tool.
Testing Ember Applications: First Steps by João Farias, That’s a bug João talks about his previous experience testing using Ember which tests the front end code, rather than the UI (which Selenium does).
Link to the comments can be found here. Thanks for sharing João, much appreciated!
I currently have a 40 minute commute each morning. Generally, it is an enjoyable drive with only one point where I am slowed down by rush hour traffic. Each morning, on the way to work, I listen to a podcast episode. There are several I follow, including some about software testing which are listed in my weekly ‘What I read last week’ blog post.
Last week, I listened to the AB Testing podcast – presented by Alan and Brent. This is only the 2nd time I’ve listened to this podcast. The main reason I don’t listen to it often is because the length of an episode is longer than the time it takes for me to drive to work – and I prefer to listen to an episode in one sitting. I started listening to episode 102.
One of the presenters had previously expressed his dislike for Selenium and as a result, twitter went crazy. It would seem that there are a lot of people in the testing community who are fiercely protective of Selenium. (I confess, I’ve not seen the original discussion. If anyone can send me a link to include, I would be grateful).
In my experience, there are usually 2 reasons why someone may criticize a test automation framework:
Lack of experience actually using the framework so being unable to fully utilize all its features.
Wanting to promote a different framework
It would appear that their reasons for disliking Selenium are neither of these. In episode 102 of the AB testing podcast, Alan and Brent expanded on their views a little more. They put forward some interesting points about the industry infatuation with UI testing. One point they made was that UI testing should be a last resort when the software is untestable. They also suggest that the industry obsession with UI testing is a result of the separation of developers and testers. There is no incentive for the developers to make the code testable in the first place, meaning that there is no choice but to resort to UI testing.
Alan and Brent appear to be more against UI testing in general rather than Selenium. However, Selenium has made it easier for novices to write automated tests without fully understanding how they work.
Personally, I wouldn’t be so quick to completely dismiss UI testing. However I do admit to being guilty of relying on it too much. I am aware of this issue and am hoping to work more on integration and unit testing in the future. With regards to UI testing, I believe that it is okay to use tools like Selenium so long as you fully understand what is going on in the code. Observe the code, break it down, understand it, and made any changes necessary. It should be understandable and maintainable.