I was revisiting my code from Day 21 of the 30 Days of Tools challenge. This involved clicking the button on the following website: https://ministryoftesting.github.io/the-button/#

In my code, I used GetAttribute(“src”) to check if the src attribute contained the words red or green to indicate the current colour of the button. Because it took about 5 seconds for the button to change back from red to green, I added a 4 second wait until the test checked the colour again.

Assert.True(button.GetAttribute("src").Contains("green"));
button.Click();
Assert.True(button.GetAttribute("src").Contains("red"));
Thread.Sleep(4000);
Assert.True(button.GetAttribute("src").Contains("green"));

It was pointed out to me that adding sleeps are inefficient. It is often tempting to add sleeps in to deal with timing issues. I decided to take another look at the test to see if I could improve its efficiency.

To start with, I decided to create 2 separate locators for the red and green buttons. Instead of checking the src attribute of the button, I checked that the separate buttons existed.

IWebElement button = Driver.FindElement(By.Id("buttonImage"));
By greenButton = By.XPath("//img[contains(@src, 'green')]");
By redButton = By.XPath("//img[contains(@src, 'red')]");

Assert.True(Driver.FindElement(greenButton).Displayed);
button.Click();
Assert.True(Driver.FindElement(redButton).Displayed);

I then uses WebDriverWait to wait for an expected condition. After the button is clicked, the test waits for the green button to appear.

WebDriverWait wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(10));
wait.Until(ExpectedConditions.ElementExists(greenButton));

I then decided it might be useful to check that it took less than 4 seconds for the button to change from red to green. I did this by recording the time before the button was clicked, and after the button had changed back to green. I then checked that the time difference between the 2 numbers was less than 4 seconds.

I still kept the timeout for the wait at 10 seconds as I wanted to confirm that the button colour did change before checking the time it took for the button to change.

DateTime dateBefore = DateTime.Now;
...
DateTime dateAfter = DateTime.Now;

double diff = (dateAfter - dateBefore).TotalSeconds;
double maxTime = 4;

Assert.True(diff < maxTime);

Final Thoughts

UI tests are, and always will be, notoriously flaky. Many are tempted to solve this issue by inserting sleeps or pauses before steps that are known to be particularly flaky. This is bad practice as it wastes time unnecessarily. In comparison to other types of automated tests like unit or integration tests, UI tests take a long time to run. As more sleeps are added to the tests, more seconds are wasted unnecessarily. This is why waits are a much better way to handle test flakiness.

However, my original solution to the challenge didn’t include a 4 second pause due to timing issues or flakiness. It was included because it was known that the button took about 4 seconds to revert back to its initial state. For this reason, I don’t think it was a bad thing that I chose to include the pause. However, when I updated the code to use waits instead of sleeps, I noticed that the button actually took about 3.2 seconds to change back. This means 0.8 of a second was wasted. This isn’t much of an issue for a solution with just a single test, but imagine if this was 1 test in a larger test suite. Imagine, if each of those tests were wasting fractions of a second. That time would really add up.

UI tests are already slow to run. We should aim to reduce the time as much as possible.

Full Code Sample

For this code to work, you’ll need to install the following NuGet packages:

  • Microsoft.NET.Test.Sdk
  • NUnit
  • NUnit3TestAdapter
  • Selenium.WebDriver
  • Selenium.WebDriver.ChromeDriver
    [OneTimeSetUp]
    public void Setup()
    {
        Driver = new ChromeDriver(Environment.CurrentDirectory);
        Driver.Navigate().GoToUrl("https://ministryoftesting.github.io/the-button/#");
        Driver.Manage().Window.Maximize();
    }

    [OneTimeTearDown]
    public void Teardown()
    {
        Driver.Quit();
    }


    [TestCase]
    public void CheckButtonWorksAsExpected()
    {
            IWebElement button = Driver.FindElement(By.Id("buttonImage"));
            By greenButton = By.XPath("//img[contains(@src, 'green')]");
            By redButton = By.XPath("//img[contains(@src, 'red')]");

            Assert.True(Driver.FindElement(greenButton).Displayed);

            DateTime dateBefore = DateTime.Now;

            button.Click();
            Assert.True(Driver.FindElement(redButton).Displayed);

            WebDriverWait wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(10));
            wait.Until(ExpectedConditions.ElementExists(greenButton));

            DateTime dateAfter = DateTime.Now;

            double diff = (dateAfter - dateBefore).TotalSeconds;
            double maxTime = 4;

            Assert.True(diff < maxTime);
    }

Further Reading

Ministry of Testing Club discussion page
See how others handled clicking the button as part of the 30 Days of Tools Challenge.

My original blog post on testing the button
See my original blog post on how I decided to click and test the button.