Creating a useful UI regression suite

Extracting as much value as possible for your selenium testsuite

Posted by Krystian Wojcicki on Saturday, March 23, 2019 Tags: Programming   3 minute read

Recently I was setting the foundation for my teams WebdriverIO (Selenium based UI testing) test cases. While developing these my main priority was of course extracting as much value from these tests as possible, but also keeping them maintainable and easy to write.

Here are some rules I tried to follow that I believe really helped keep the test framework clean and reusable. Some of these have specific syntax related to WebdriverIO but the general advice will apply to most Selenium based testing.

  • NEVER use driver.pause use driver.wait/driver.waitUntil. Using driver.pause is a great wait of keeping your tests flakey and none-deterministic. Waiting for 1 second may be enough for the UI to load on your machine but may require 2 seconds for a college etc. In addition keep the timeout generous, these tests are meant to test UI functionality not stress tests or backend response times.
  • Watch your tests. This is very important while writing tests to ensure your tests are working as expected. But its also necessary to watch test runs later on, ensuring some regression hasn’t happened that your testcases aren’t catching.
  • Make your UI easy to test. Relax with the funky animations and flying divs. Keep it simple and easy to use not just for yourself but you for users. In addition ensure your elements have id tags this will make creating + maintaining xpaths much much easier.
  • Watch out for animating components. Sometimes animations are necessary or very helpful. Encapsulate any testing logic around animations ensuring its airtight, isn’t relying on driver.pause.
  • Infinite textareas/scrolling. Scrolling in custom elements such as code editors can be a pain. Similar advice with respect to animations. Create methods for writing to textareas, reading from textareas etc, ensure all testcases use these methods making it easy to fix any bugs related to these components.
  • Write a test plan. This will help keep track of what parts of the UI are under tested, what is being tested and how.
  • Take screenshots/record tests. Not only will this help in debugging when a testcase fails it also helps immensely when creating documentation. Have a new feature that requires documentation? Hit 2 birds with 1 stone. Write a test that takes screenshots showing how to use your feature.
  • Assert often and everything. Similar to regular unit testing, asserting everything and often helps catch any bugs that may occur.
  • DRY (Don’t Repeat Yourself). It is tempting to take a test, copy it and replace some small bits to test your feature. That isn’t to say create methods for everything, but as with regular programming some balance needs to be struck.
  • Table driven tests. This is a great way to code tests as it will force methods to be more generic and more useful. Heres a great tutorial about table driven tests. While the language shown there is Go its applicable to any language. If that testing paradigm isn’t right for you, then one can check out behavior driven testing and mocha-steps