APM Is Dead – Long Live APM!

It’s a busy time for Application Performance Management (APM) vendors. The last couple of years have seen a flurry of activity with private equity takeovers of long-standing companies like Compuware, Keynote and BMC, venture capital infusions into rising stars like New Relic and AppDynamics, acquisitions of niche vendors like Opnet and CopperEgg, and the emergence of new players, including Exoprise, AppNeta, and ThousandEyes.

If investment activity is any indicator, these are clearly signs that the APM market is poised to expand significantly.
 
And yet, most of the solutions on the market today are built to support what is essentially the same types of users as they always have, IT departments running apps on systems they manage or web/cloud app providers (e.g. Twitter, Salesforce.com, etc.) hosting apps and services for public consumption. The development operations (a.k.a. “dev ops”) and network ops teams responsible for application service levels have come to rely on APM solutions as a natural extension of the application development cycle, providing continuous performance feedback both in the lab and in production.

Much of the interest and activity in the APM market has been fueled by the rise of cloud computing, particularly in three key areas.

• The Cloud App Explosion – Is your team building a killer cloud-based app? Of course! Who isn’t? Take a scroll…okay, several scrolls through the AWS marketplace and it’s easy to see how far the proliferation of cloud-based apps has come. Behind every one of those logos is a development team half praying for, half panicking at the prospect of having thousands of concurrent users…and whether or not their app will stand up to the load.
   
• Cloud Ready APM – To support these “born-in-the-cloud” apps, APM solutions have had to evolve to support some of the unique characteristics of apps hosted on IaaS platforms like AWS and Azure, providing compatibility with, and visibility into the hosters’ infrastructure and platform services.
   
• Cloud Delivered APM – At the same time, the market demand for cloud-based apps has also had implications for the APM solutions themselves. APM customers, especially those developing cloud-based solutions, don’t want solutions that require them to provision and manage servers. New Relic, for example, has built a very successful business by ONLY delivering their APM solutions from the cloud.

But wait. Something’s missing here.

Sure, there are a lot of apps being developed and hosted in the cloud, and those teams need to monitor and manage app performance, but what about the customers? Aren’t there a lot more of these SaaS app “consumers” – IT and business operations teams that purchase and maintain portfolios of cloud apps for their organizations – than there are SaaS app “producers?” And don’t those IT teams still support users who expect them to maintain high application service levels regardless of where the app runs?

Absolutely. Unfortunately for these these teams, however, most APM solutions on the market today aren’t well suited to their needs.

This is why many organizations, even companies like Adobe, Intuit, Conoco, and ConAgra, who already have a myriad systems management and monitoring tools, are seeking alternative solutions to help them manage their cloud-based apps. It’s this gap – this emerging APM customer base, IT business operations, and their significantly different needs – that will disrupt and reshape the APM vendor landscape in the years to come.

Different Strokes for Different Folks

Nobody in their right mind would think that a source code debugger would be a useful tool for an IT team managing a Microsoft Exchange Server farm. It’s the wrong tool for them. It might be able to provide a ton of information, but it won’t be information that’s useful or actionable by the IT team.

Likewise, you can’t expect an APM solution, built for dev ops teams, to work well for a business ops team consuming Exchange Online, Dropbox, Salesforce.com or any other “black box” SaaS app. These teams can’t log into the app servers, they can’t instrument the application code, and they can’t directly access log files or SNMP messages from the majority of the network infrastructure that connects them to their cloud apps.

The fact is that the APM needs of business ops users are fundamentally different than those of dev ops. Here’s why:

Service Level Management v. Application Tuning

By definition, dev ops is the combination of application development and operations concerns, with the goal of providing a feedback loop that assists both developers and operations personnel in optimizing the delivery of an application or service they manage. These teams use APM tools to let them know which specific code or infrastructure “knobs” to turn to improve their application performance and reliability.

By contrast, business operations combines the concerns of particular groups of users (e.g. sales, marketing, and for certain apps, the entire organization) and IT. These are cloud application “consumers” rather than owners. They are focused on keeping their users connected to and productive with the apps they rely on. However, because they leverage multiple app and internet service providers, business ops teams need a different set of tools that help them verify and manage service level attainment across multiple vendors in addition to detecting and isolating problems within their own network.

To survive in this diverse cloud app environment, these teams need to be able to answer several questions:

• If there is a problem with an app, is it caused by something in my own network, something in the ISP network(s), or something at the app provider?
   
• If it’s a provider problem, what details can be collected to help with escalation and expedition of a support case with the provider?
   
• Are my app and internet providers meeting their service level agreement (SLA) commitments? If not, can I collect evidence that will support a refund request?
   
• How do my application performance numbers compare across locations, apps, and vendors? How do my numbers compare to other application customers?

Hands-off v. Hands-on

Perhaps the most obvious difference between a dev ops team managing performance for their app(s) and an IT / business ops team managing performance for an app like Salesforce.com is level of access to the application source code and hosting infrastructure – for these 3^rd party apps, business ops teams have none.   The apps these teams manage are completely “black box,” as are most of the app delivery networks they rely on to access them.

For this reason, solutions that require code-level instrumentation, or even tight integration into the app delivery network, are impractical. Business operations teams need APM solutions that can effectively function solely by interacting with the public facing UI’s and API’s provided by the public cloud apps.

Lots of apps v. Lots of users

Application dev ops teams, particularly those building consumer or B2B apps hosted in the public cloud, are usually focused on a single application or a relatively small set of apps they build and manage. However, they are trying to test and optimize their app delivery for a nearly infinite set of users, remote end points, and code execution paths. They want to gather and analyze as much data as they can from this vast, unknown, set of users and locations, without degrading the user experience. Again, this is why solutions that work through injection at the hosting point of origin make so much sense.

Business operations teams, by contrast, have a different problem. They generally have a relatively finite, and well known set of users and points of access they are managing, but need solutions that enable them to manage a wide, and growing, array of apps from multiple vendors, without requiring them to become experts in the protocols and syntax for each and every app.

Inside-Out v. Outside-In

If you are the application hoster, you want to have data that reflects the performance of your application from points outside your network. After all, that’s where your users are, right? Whether you use one of the solutions that synthetically monitor from points of presence (POPs) in the cloud or use a passive/real user monitoring (RUM) solution to infuse tracking code into the app, as a dev ops team you are generally interested in an “outside-in” view of application performance.

Business ops teams need to look in the other direction.   For most, the bulk of their users access apps out in the cloud from inside the office network. Monitoring solutions that operate out of vendor managed POPs aren’t as effective in these cases because they don’t exercise that critical “last mile” spanning from the ISP/access provider through the organization’s own network segments. This is a big gap, as these last mile components are very often the source of application availability and performance problems. If you are only monitoring your cloud apps from the cloud, you have little chance of detecting and resolving problems before they impact your users.

Ease of use v. Depth of analysis

Service providers can draw a clear business case to justify investment of resources into the integration, deployment, training, and ongoing management of an APM solution. If you don’t deliver a high quality user experience, users won’t continue to use the application and/or you won’t be able to effectively scale to support large numbers of users. As development organizations application service providers also have the skills and fluency with their own application needed to effectively integrate an APM solution and interpret the detailed data it provides.

Business ops teams are focused on the needs of their organization and users. APM is a means to an end for them and they need to do it as effectively as possible, with as little investment in time, money, and infrastructure resources as possible. These are true operations teams, not software development teams. For this reason, solutions that require complex integration and/or scripting become too cumbersome to manage, especially as the organization’s application portfolio grows. The cloud apps that business ops teams leverage are becoming increasingly user friendly and easy to manage. They need APM solutions that are equally so.

Business ops teams need solutions that provide a broad analysis, both in terms of the number and diversity of apps they monitor but also the end-to-end view of their network path between their users and the cloud apps themselves. They aren’t looking to shave 100 msec off of a particular application’s login time. They are trying to detect when critical user transactions that should take a couple of seconds start taking 10 or 20 seconds, and if they do they need to be able to pinpoint the the cause of the problem, even outside their network, so they can take effective action.

The Future of APM

Bus Ops is the New Dev Ops

Clearly business operations teams using cloud-based apps and services have application performance management needs that differ significantly from those of application service providers and dev ops teams. While there are a number of solutions on the market that categorize themselves as “Enterprise APM,” even these solutions tend to be oriented more toward teams managing performance for apps they themselves operate.

They’re still application performance management, but maybe the solutions for business ops teams are so different, they should be considered a completely new category – “APM for Business Ops.” Is this a niche? Up to now, perhaps. But with Amazon, Google, Microsoft and other big players fully committed to the cloud, all indications are that cloud apps and services will continue to gain significant share of application portfolios in organizations of all sizes.

So while the APM for Dev Ops wave seems to be cresting, behind it looms a much larger APM for Business Ops wave. How much bigger? To get an idea, just think about the ratio of cloud app consumers v. cloud app producers.

APM vendors who want to ride this wave will need to do more than update their marketing.