Real User Monitoring vs Synthetic Monitoring: Why You Need Both

When it comes to Application Performance Management (APM), there are two main technologies used to measure end-user perspective performance: synthetic monitoring (STM) and real user monitoring (RUM).

We’ll discuss the pros and cons of each of the technologies especially related to monitoring third-party apps. Third-party apps like Office 365 and GSuite are experiencing a surge of growth for mission-critical purposes within the business.

What is Synthetic Monitoring?

Synthetic monitoring, also called synthetic transaction management (STM) or active monitoring, uses pre-recorded scripts to emulate the actions a user would execute against an application or web app. Synthetic monitoring enables DevOps and IT staff to proactively measure the availability and performance of applications and infrastructure. Synthetic transactions can execute 24×7 and let teams know if there are any problems, end-to-end, with an application or dependent infrastructure.

You can think of synthetic transactions as a robot that is constantly monitoring an application as frequently as once a minute.

With Exoprise CloudReady, there are sensors that cover most mission-critical cloud third-party applications like Office 365, GSuite, Service Now and Salesforce. And these sensors fully test the applications, Single Sign-on, uploads, downloads, network path performance and more. And because the sensors are maintained by Exoprise, customers don’t have to maintain scripts or keep up with application modifications. Exoprise does that for you.

In other cases, CloudReady sensors are executing pre-recorded actions against APIs and, in the case of Skype for Business, automating the actual Skype client for the synthetic tests.

High-Level Actions, Low-Level Metrics

We like to say that our sensors are executing high-level actions and capturing low-level metrics from any of your locations where you want to monitor. Additionally, because all of the Exoprise customers are running the same or similar pre-recorded actions, it enables us to crowd-source the data for cross-tenant comparisons, application and network benchmarks. This is very powerful for understanding whether your end-user experience is better or worse than other customers and environments.

Benefits of Synthetic Transaction Monitoring

• If you are using synthetic transaction monitoring, you will be notified of an application outage first, before a real user, and this can help you respond faster and notify your operations and support staff.
• Synthetic monitoring allows you to monitor the performance and reliability of an application 24 x 7, even when no users or customers are accessing the app
• Third-party applications or SaaS apps like Office 365 are increasingly becoming the majority of the applications IT delivers. Synthetic monitoring for these apps is often the only available insight as third-party applications can be instrumented or altered by customers.
• With synthetic monitoring, it’s often easier to establish benchmarks for application and network performance. Since synthetic tests run all the time at regular intervals, the baselines produced by synthetic testing are more accurate than the performance data that is returned from real user monitoring which tends to be sporadic.

Limitations of Synthetic Monitoring

There are some limitations to be aware of when it comes to synthetic monitoring. Synthetic monitoring does not tell you what your actual users are experiencing. In some cases this may mean that you believe the application is performing well but you have users somewhere that are being affected by poor networking. Other synthetic transaction limitations include:

• Problems with specific browsers
• Problems with parts of the application or API that are not being tested
• Geographical problems where you don’t have coverage for a network or application
• Problems with a specific user’s desktop
• Intermittent problems with a network or application stack that may be difficult to reproduce with synthetic monitoring

What is Real User Monitoring?

Real user monitoring (RUM) is a method to monitor how actual end users interact with an application. Also known as a real-user measurement, passive monitoring, real-user metrics, or end-user experience monitoring (EUEM), this type of monitoring continuously and passively captures the system in action, tracking errors, functionality and performance characteristics of the app.

In contrast to synthetic monitoring, real user monitoring is unscripted and captures everything that a user does with an application in real time. There are various ways to capture real user activity for an application depending on what kind of access to the app is available:

• Passive EUEM can be captured at the network but this typically requires SSL inspection or man-in-the-middling because most apps are now behind SSL security.
• If you have access to the servers and application stack, Javascript injection can capture the performance of a web page but this is often difficult to accomplish when the issue may exist across different servers or third-party web/micro services. All of the different servers will need to be instrumented and the data collected in a single location.
• Through browser extensions such as Service Watch. Uniquely, Service Watch can monitor the performance of a real user’s interaction with an application, doesn’t require SSL man-in-the middling or expensive network capture.

Benefits of Real User Monitoring

• With RUM, you capture the experience of a user no matter where they are. If you have a user that is roaming or telecommuting and they are experiencing poor performance, you may not have synthetic transaction monitoring in place to detect the issue.
• Individual users that complain of slow performance can be ideal candidates for monitoring using Exoprise Service Watch.
• Real user monitoring like Service Watch can be ideal for monitoring and focusing on critical line-of-business apps (LOB) like Salesforce, that may be mission-critical for a subset of users.

Limitations of Real User Monitoring

• Real user monitoring (RUM) can’t tell you anything about your systems or, more importantly, third-party cloud and SaaS Apps when you don’t have any user using them. This may not be as critical for systems you own, because you probably have infrastructure monitoring in place that is telling you if have a problem with servers. But for systems that you don’t own – like Office 365 or Salesforce – you want to know ahead of time whether those systems are having problems.
• For you to know about a problem with RUM, one of your users has to have experienced the problem. It’s better to find out about problems with synthetic transaction monitoring and it affecting a robot user rather than an outage or problem affecting a real user.

Combining Synthetic Transaction Monitoring with Real User Monitoring

Combining synthetic and real user monitoring gives a complete view of the user experience along with a high level view into the application and dependent networks. Real User Monitoring can miss outages, page errors, and third-party problems. While Synthetic Transaction Monitoring is only a proxy for real-user experience and can miss problems experienced by various user populations in areas where you may not have STM coverage.

With Exoprise, the whole really is greater than the sum of the parts. Using the best synthetic and real monitoring together provides a complete view of the system, infrastructure, and user experience. Using both techniques in tandem eliminates the inherent blind spots of each technique and can provide an organization with the best view of their users’ experience.

Also, check out Forrester and Exoprise video on why both are needed:

Benefits of Using Synthetic and Real Monitoring Together

• Eliminates finger pointing
• Reduces Mean-Time-To-Resolution
• Understanding impact on actual users allows for better business decisions
• Brings business owners into the discussion

A unified system combining both synthetic and real monitoring not only facilitates these organizational changes but if it’s implemented well, provides much greater ease of use, a single system to learn, and data combined and displayed in a common manner.