With every webpage loaded, email sent, or video streamed, network traffic takes a complex journey…
Selenium is a great open source project that can be useful for monitoring simple multi-page web apps, but using Selenium for testing or monitoring SaaS services can be a challenge. Occasionally, our sales and engineering teams talk to prospects who say, “We have other fellas who write Selenium scripts, I’m just going to ask them to write tests for SharePoint Online, Microsoft Teams or Exchange Online”. We get it: you’re trying to save your company money. Or not invented here (NIH) syndrome. Or “our environment is too special and different, we need something custom”. We’ve got you, been there.
Read the rest of this article to figure out why it’s hard to use Selenium for testing cloud services and SaaS properties like Office 365. In the end, your organization won’t get the benefit and it will be more costly to write your own Selenium scripts for monitoring cloud services.
Quick Primer On Selenium
Selenium was originally developed in 2004 by Jason Huggins as a quick testing tool for HTML/Web pages. Commercial tools at the time were insufficient, overly complicated, and mostly designed for thick-client apps. Selenium enabled easier automation of web browsers to test and execute web pages, submit forms, click buttons/links and generally navigate around.
Selenium, at the time, was great because it didn’t require learning a new complex scripting language or environment. Web developers could figure out how to put together tests with JavaScript and simple HTML.
Selenium is one of the most popular automation testing frameworks. Being a free, and open-source framework has made it a popular choice among the global testing community. Selenium over time has gained critical support of multiple programming languages and browsers making it a perfect choice to perform automated cross browser testing.
Here’s what sample Selenium script looks like:
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("https://www.exoprise.com/freetrial/");
driver.findElement(By.linkText("Sign Up Now!")).click();
driver.findElement(By.className("mycheckbox"));
driver.findElement(By.linkText("Login")).click(); //using Selenium click button method
No too bad but not the prettiest and this script doesn’t do anything except click a button and wait for a form. It doesn’t fill anything out, it doesn’t performance monitor anything, it doesn’t upload, it doesn’t send an email. You catch our drift.
Its Not You Selenium, Its The New Web
Its not that Selenium is bad, its that the web and web applications have gotten more complex. Web apps like Outlook Web App, SharePoint, OneDrive, and, whoa, Microsoft Teams look and behave like real desktop applications. These applications, sometime referred to as Single Page Apps (SPAs), are complex to build, maintain and there are subtle changes all the time.
The SPAs are driven by background iframes, AJAX requests, WebWorkers and all kinds of new stuff in browsers to make the web and Internet a richer and more flexible development environment. And these things are hard for Selenium to deal with, test, automate and keep up with. It becomes more difficult when the apps change all the time. You’ve got to be really good at Selenium scripts and understand the way browsers work, asynchronously (in the background), and how it all comes together.
That’s what we do here at Exoprise; we build automated tests at scale to monitor web services and apps on behalf of customers.
Web Monitoring with Selenium Can Be Brittle
We’ve established that Selenium is a remarkable project and open source solution for testing lots of web pages and simple web apps. But for complex SPAs, like those typically delivered from cloud vendors like Microsoft, Google, and Salesforce, Selenium starts to fall down. Here’s a few reasons why Selenium is brittle and can be difficult to master for complex Web apps:
Finding Elements Is Tricky for SaaS Apps
Locators in Selenium lingo are the way you find elements on a page, aka selectors. The buttons, the input fields, you find them so you can assert their presence and interact with them. Choosing and building the right locators (they are often hierarchical) for Selenium so they last and won’t suffer disappearing through a version or change is a skill that takes practice and experience. And locators are often specific to the framework and development environment of the web app.
Locators for SaaS applications can be even trickier. Large scale, popular SaaS services like Microsoft Office 365 support lots of different versions, editions, customization’s, and languages. That brings very dynamic usually generated on the fly and that means that locators will often be transient, ephemeral, or just change frequently. This translates into very difficult web pages to test and monitor for correctness, delivery and performance.
Once You Find The Element, Interacting Is Difficult Too
Half the battle is finding an element. That helps you test that the page was properly delivered and materialized. Now what are you going to do once you find the element? You want to click on a link, fill in an input form, click a button. That can be difficult too for various reasons.
- Asynchronous technologies
Most of the newer JavaScript frameworks such as React, Angular, and Vue.js are designed to make the updating of web pages with background loading of data and fluid changes. This background updating and dynamic re-display/re-rendering can be particularly tricky for Selenium synthetic monitoring to deal with. And it is the cornerstone of making web pages faster to update or appear so.
- Background updating makes for lots of waits and DOM settling
With all of the background updating and the dynamic nature of these new fangled web pages, there’s lots of subtle waiting required to make testing and navigating pages difficult. And if you’re trying to performance monitor the page for anomalies then you have to balance waiting for elements vs finding errors and problems on the page.
With these newer asynchronous technologies comes a very responsive Web page and app. Data can be retrieved in the background and the whole page can be morphed and changed. For example, clicking on a button within a page can involve the following steps and measures when using Selenium:
- First, you have to wait for the element (button) to be created on the page. For many Single Page Apps, the button won’t statically exist in the delivered HTML but created dynamically in response to data or other interactions with the user. Usually, the button is created from background JavaScript, XHR, or from within script that is executing on IFrames. These elements can be tricky to observe and track down.
- Even after the element is created, it may be disabled or not visible. Usually this is an optimization from the framework and it can prevent Selenium from interacting well with the page. These optimizations can vary by Javascript framework and browser.
- The element can be obscured behind other elements as an optimization or due to varying page size and media render settings.
- Finally, we’ve even seen cases where an element is positioned off-screen or not within the viewport of the browser. This also varies by browser and framework.
That’s a lot of factors just to click a button.
How Exoprise Tests & Monitors Web Services
So how does the team at Exoprise test cloud services and avoid some of these Selenium issues? To start, we don’t currently use Selenium. Instead, we use proprietary headless environments and “drivers” that are more closely tied to each execution unit.
Our proprietary browsers are still driven by JavaScript which is important because JavaScript is often required to be “injected” into the page (minimally) for automation as well as driving and detecting state in the browser and SPA.
There are other differences as well:
- Deep understanding of headless browsers
Since the web has become more asynchronous and browsers/JavaScript more capable, the team at Exoprise is more dependent on understanding the internals of browser environments and how they interact with Single Page Apps like SharePoint or GSuite. - Eat our own dog food
We utilize most of the SaaS services we monitor and continuously run our own sensors in various development, stage and production environments. This enables us to detect problems or updates and fix monitoring scripts hopefully before the changes are encountered by customers. - Deep understanding of the applications we are performance monitoring
For complex apps like SharePoint, Salesforce, or Microsoft Teams, we have a deep understanding of the technologies (WebRTC) and frameworks utilized by the app. This enables us to write less brittle monitoring scripts that interact with the application. - Rapid updates
CloudReady has infrastructure that supports updating the scripts in real-time if they have to be. Usually, monitoring scripts and environments can be securely signed and updated within 1-2 hours across the environments. This enables us to quickly fix issues and deploy updates if required. - Lots of experience choosing locators (selectors) using CSS, XPath, Regular Expressions, and more
Choosing locators that stand the test of the time is one of the keys to writing robust scripts that to monitor SaaS services. The development team at Exoprise has experience choosing the right locators that often survive application updates, language changes and versions. - Low-level plumbing
Finally, because our browser environments are proprietary we can leverage lower-level browser state from the driver scripts that Selenium wouldn’t give us access to. This lets us understand more about the asynchronous state of the browser and web application content and what is making the application slower.
Try Exoprise for Web Monitoring
If you’re trying to write your own Selenium scripts to monitor Office 365 or other web applications, we invite you to give Exoprise a try. We think you’ll see that it’s an easier and more compelling way to monitor the web and Internet. And there are plenty of other benefits too:
- Crowd-powered
If you write and host you own Selenium performance monitoring scripts, you won’t get the benefit of our crowd-sourced monitoring. Read more about the benefits of crowd-sourced monitoring. or download a whitepaper about it. - No servers required
With CloudReady you don’t have to deploy or maintain any servers. Our monitoring sensors can run from any spare desktop, MiniPC or virtual machine. - Your place or ours
With Exoprise, you can run the same scripts inside your firewall as well as outside in public clouds. Monitoring SaaS services from both locations is super critical.
Related Posts
