You don’t have to get rid of Selenium to get rid of your Selenium problems

It’s clear that Selenium is the most popular choice for organizations automating their UI testing, but teams are still running into challenges with it. Learn how to solve those challenges without abandoning the framework altogether.

When we ran surveys on UI testing tools, it became very clear that Selenium has become the de-facto standard. But the odd thing was, no matter who we asked, we always got the same caveat: “We love Selenium, but there are challenges.”

So we went back to the community and asked, “What are your top UI testing challenges?”

The results were clear: the number one challenge was building tests that would be maintainable over time. Through further client interviews, we distilled the major Selenium problems down to these five challenges:

1. Creating reliable locators and wait conditions
2. Maintaining tests when the UI changes
3. Debugging and diagnosing automation failures
4. Time taken to execute entire suite
5. Knowledge/skills to create scripts

Then, we started looking at how we might be able to help solve them without having to get rid of Selenium. Turns out, you can solve all of these challenges by adding Parasoft Selenic into the mix — but there are lots of relatively simple ways to solve all of the challenges of Selenium, without abandoning the platform that everyone clearly loves.

How to solve the top 5 challenges with Selenium

So let’s dig deeper into these challenges, and how to solve them.

Challenge 1: Creating reliable locators and wait conditions

Creating reliable locators involves understanding how to build an identifier that interacts with the right element every time, and is stable between different executions. Users find themselves digging through Chrome developer tools to find the right HTML block, figuring out what it is doing, then spend time building the smartest locator possible. Usually what ends up happening is that the element locator is selected in the application when it’s in the current state and that on playback things tend to be different. This can be really frustrating.

Wait conditions play a huge factor here too. With modern web applications, loading content dynamically within a single page means that even if you find the right locator, you have to be smart about building the correct wait condition to ensure the element is ready to be interacted with. It can all be a huge pain.

The Solution: Smart Locators

There are many UI testing solutions out there that are focused on solving this exact challenge using the term “Smart Locators”. Smart Locators capture the most intelligent locator possible when initially building the test script so that it’s more resistant to change. My recent post that calls out 10 great new tools for UI testing includes several tools have record and playback capabilities for this.

Additionally, there are plug-ins for Chrome that can help during script creation to extract the best element locators right out of the page. I’ve been looking at several of these plug-ins, and by far the best one I’ve seen is TruePath.

Challenge 2: Maintaining tests when the UI changes.

This is a painful challenge that organizations face when it comes to ongoing Agile testing activities. We hear this story time and time again — testers spend a lot of time building their Selenium test (with good locators and wait conditions), and then the application UI changes and their tests break, often failing the nightly build. The tester then has to go back and figure out what changed in the application, identify whether it is a real defect / regression or whether it’s just something wrong with their test script. And then go through the process of fixing or updating the test by identifying and creating new locators / wait conditions. It’s a vicious cycle.

The Solution: Artificial Intelligence

AI can really help you in this area. Yeah yeah, I know, AI is such a buzz word these days. But there are extremely practical usages. In particular, Parasoft Selenic uses machine learning during Selenium test execution to solve this exact challenge. Each test run produces a unique entry into its history, showing the current state of the DOM and indicating whether the test was successful or not.

If the test were to ever fail due to application UI change, Selenic goes back through time and tells you how to construct a new locator that will work with the new version of the application. This is really powerful because it means you don’t have to leave Selenium to overcome maintenance challenges from changing UIs (and an added benefit to this technology is its ability to self-heal your CI/CD pipeline, automatically updating the locator at runtime to stop the build from unnecessarily failing).

Challenge 3: Debugging and diagnosing automation failures

Debugging and diagnosing Selenium tests is difficult and can be a considerable time sink (especially when coupled with an ever-changing UI). We found that various solutions had sought to solve this challenge by providing their own frameworks with advanced reporting features, but then, testers ended up being locked inside a proprietary platform or IDE. This is “ok” if you plan to stay with that vendor or tool for the rest of your life, but what about Selenium users who want to keep the original freedom that open-source provided them?

The Solution: Intelligent Automation and Reporting

To solve this challenge, you can use a combination of reporting and analytics of the test execution, coupled with IDE tools.

By plugging reporting tools, like Parasoft Selenic, into your existing CI/CD pipeline, you can more easily understand which tests are failing and why. With runtime self-healing, Parasoft Selenic can prevent the build from unnecessarily failing due to unstable tests. After the testing is complete, Selenic will provide recommendations to fix those broken tests, which can be imported directly into the IDE for automatic fixing.

The IDEs themselves, such as Eclipse and IntelliJ, have debugging capabilities that allow you to set breakpoints in your Selenium tests and pause the application at a particular point, so you can pull out information about the state of the UI and what might have because the application failure.

Challenge 4: Time taken to execute the entire script

Automated UI tests take longer to run than tests that run against the API/service layer of your application (and much longer than isolated unit tests), and the more UI tests you create, the more time that it takes to execute those tests… and the longer you have to wait for feedback from the CI/CD pipeline. The trouble is, the pipeline is getting triggered multiple times a day due to frequent code changes, and you cannot wait until the next day (or sometimes over the weekend) before you get useful feedback. There needs to be a way of optimizing the execution of your test suites that balances quick feedback with comprehensive coverage.

The Solution: Test Impact Analysis

This is where you can leverage technologies such as Parasoft Selenic’s test impact analysis. By tracking the code that each Selenium test covers, and correlating that with the changes in the code base, you can prioritize the subset of tests that need to be executed to validate the code changes. With Parasoft Selenic, you don’t need to have access to the underlying code because it simply scans the binaries (e.g. .war or file), identifies which classes have been changed between builds, and then maps that to the affected test cases to enable you to run a subset of your Selenium tests as part of the CI/CD pipeline.

Challenge 5: Knowledge and skills required to create scripts

Lastly, the knowledge and the skills required to create the Selenium tests in the first place can make Selenium usage prohibitive. Depending on who we talked to, this was either the first or the last on the list of challenges people were facing. We would typically see that when users were just starting out with Selenium, their number one challenge was test creation — but once they were going and starting to build a mature testing practice, it was no longer the number one challenge, and really the other problems became more important. So the way I see it, getting over that initial hurdle is the real challenge.

The Solution: Code Helpers

There are such things as code-less Selenium and NLP driven Selenium creation, but I haven’t particularly seen one that works the way we would all hope it would. One of the major realities of Selenium is that it’s code. This is good and bad. It’s good because if you have things that you need to do that are outside of the “normal practice,” you can find some good code examples by doing a quick google search. It’s bad because it’s code, and that represents a barrier for some testers.

So, do you need to be a developer to build Selenium scripts? The answer is no, but some development skills do help. As I mentioned earlier, many of the great new UI testing tools have recorders that will build Selenium scripts, such as Parasoft Selenic, SeleniumIDE, and Katalon. These helper tools will give users who are new to Selenium a jumpstart on the practice, and provide a middle ground between code-less and coded. Of course, you have to be aware that no UI record and playback tool is perfect (of course, there are ways to solve those challenges as well).

Conclusion

Selenium is the #1 choice for UI automation, the technology is getting better every day, and there are many technologies out there that are built around supporting Selenium and helping solve some of its basic challenges. UI testing is critical to validating the customer experience, and being able to build rich and meaningful Selenium tests, that are maintainable over time, will help increase the overall test coverage in our industry to build better, more meaningful experiences in the future.