Skip to content

The SaaS APM Racket – Collect, Relay, Inform, Mine and Extort (CRIME)

jinspired.logoThe near $1B valuations given to some SaaS based application and network performance monitoring vendors might appear at face value to be absurd until you factor in the long term aspirations of these companies which is to be more than a performance monitoring solution vendor by collecting vast amounts of data related to your critical operations, transactions and user activities, and then selling back access to the very same data (and more) at a premium charge.


Todays SaaS APM solutions, such as NewRelic and AppDynamics, offer very limited functionality when compared with their counterparts in the enterprise space but this is largely by choice so as to make it as simple as possible to net customers, penetrate operations (with zero-config agents), collect data, gather intelligence and finally drive future premium data analysis offerings. There is no real crime as such committed here, it is done with full consent though if you were to ask many customers of such solutions what is known about the data recorded, relayed and retained they would readily admit to being completely in the dark except for what is presented in the monitoring dashboard that got them hooked in the first place (that and some data nerd t-shirt).

Clearly these solutions do appeal to the masses forming in the cloud so it is not something that is going to be eradicated by the blowing of a whistle, but we can take precautionary measures to ensure continued and unmetered access to the very data our customers produce via their monitored interaction. Here are a few of the safe guards we have been investigating in the design of future offerings with this in mind.

Externalize Data
You can obtain a very high level of protection from future data extortion (or in marketing speak value extraction) by requiring a performance monitoring vendor to persist all data collected to an external cloud storage service that you have independently commissioned. The data will still need to be stored in an open and documented format with no dependencies on private code or data libraries. No data should be retained outside of this storage except for meta data that is generated post the collect and relay stages.


An added benefit of this safety measure is that you can control (grant and revoke) access to the data across multiple vendors offering different monitoring, management and analytical capabilities. You can even get insight into how and when the data is being accessed by such vendors. And if you want to drive efficiency across the whole value supply chain you can even meter and bill access to the data. Amazon AWS S3 makes this relatively easy to setup.

We believe that all cloud services should be required to take this approach allowing (service) delegation of common aspects of a service such as storage, computing (tasks), security as well as alerting, notification, auditing, metering and billing.

Duplicate Data
Some vendors will howl very loudly having to design their data collection and data analytics engine around a generic cloud storage service interface complaining of poor performance and reliability/availability concerns especially as many don’t actually use the cloud or its services. In such cases customers can require that performance monitoring vendors provide optional plugins, or extension points, that allow duplication of the data collected to be transmitted to other vendor service points as well as an externalized storage service. Again the data will still need to be stored or transmitted in an open and documented format with no dependencies on private code or data libraries. What is important here is that monitoring data becomes secondary in the evaluation of competing vendor solutions that offer unique insights and analytics as well as integrations. It is the service that distinguishes them.


You may need to factor in the costs of such duplication if data transfers are metered though surprisingly for us most SaaS APM customers give little regard for this today. This approach also allows you to evaluate and compare as well as validate findings and insights across multiple offerings. And remember many marketed integrations across cloud offerings are driven from a business perspective and rarely a technical one, which is also why many cloud vendors with such integrations don’t use each other services to manage their own operations. It would appear it is acceptable to cross sell each others cloud wares though you would not use the service yourself.

If you want to get maximum value from data collected then consider opening up channels to multiple end points. Flow (data or execution) is likely to be incredibly important for the cloud ecosystem in the long run. Flow allows new forms of structure to be created with regard to the organization and behavior of systems, services and value supply chains.

“The constructal law provides a unifying theory of evolution. It holds that inanimate and animate phenomena generate evolving configurations to move more easily. The constructal law also provides a new definition of what it means to be alive. It states that life means flow and the free generation of design. If the flows stop, the system is dead (in thermodynamic equilibrium). The constructal law is the physics law of life and evolution” – The Constructal Law

Duplicate Software
Unfortunately it is highly unlikely that you will find a performance monitoring vendor willing to externalize or duplicate data collected by runtime agents or network packet sniffers as it increases the chance of competition along the entire service supply chain, loosens their stronghold over your data, diminishes data gravity (and its exclusivity value) as well as requiring greater transparency on what is collected beyond what is reported. To obtain some degree of protection customers can require that any SaaS based performance monitoring vendor also offer an on-premise based version (which could be in the cloud but controlled by the customer) that runs in parallel and integrated with the same data collectors. A hybrid solution of sorts.


For many vendors it would appear that greater data transparency is something to be avoided at all costs even if it means not allowing other vendors with far more efficient runtime agents to push data to them. This does not mean that data is not shared as many vendors do offer alert/event service integration but here the degree of sharing is limited.

Deprecate Data
And finally you can always try to eliminate as much as possible movement and retention of performance monitoring data collected beyond the transient scope of some execution window or process. If companies are truly agile, effective in their adaptations to environment changes, and able to influence behavior then you would expect the value of the data collected to depreciate rapidly because it would reflect a previous state now invalidated (except for maybe the last successful transitioned state). Here we replace the collect, relay, inform, mine and extort sequence with collect, relate and adapt. This new process is performed locally, within context and under control of the software itself. Instead of retaining measurement data, signals and resulting software runtime adaptations are recorded in a hierarchy of conversational scopes within the runtime and infrastructure which we call adaptation boundaries. Data is still produced, maybe even duplicated via simulation, but understood and consumed by the very same software (or execution context) that produced it and then made obsolete or recycled in near real-time. Data retention serves mainly a software training purpose. Beyond conversational boundaries the data associated with signals, emitted by the software, are meaningless and of no external value.


This feedback loop approach is the ultimate in (data) flow – fast, efficient and effortless. Once we stop being the primary consumer of performance data, by discounting candy laden monitoring dashboards that offer an illusion of control and serve to further our data addiction, we can free ourselves from being exploited by our own data ignorance and our yesteryear habit of writing on cave walls.

No matter which safe guard you choose, make sure you know the data, own the data and control access to the data. Never let data flow (life) be pushed and pulled towards a black hole from which it is near impossible to escape without paying a high cost.

“Where some see ‘coincidence’, I see ‘consequence’. Where others see ‘chance’, I see ‘cost’” – The Matrix Revolutions

Updated 5th March ’13
I’ve done a little bit of research since the publication and found the following articles which paint a similar story.

“The portability of cloud services has been the major issue of concern among the IT industries. The situations like vendor- lock-in, lack of standardized data formats and complex service level agreements are still affecting the majority of IT sector from adapting this widely emerging technology. This paper suggests a new way in order to inculcate the portability among the cloud vendors and to maintain the consumer’s trust in cloud services by ensuring that consumer is the ultimate owner of the data throughout the services. The paper proposes introduction of reliable third party (mediator) between the cloud service provider and the cloud service consumer to remove the various portability issues encountered while switching among the clouds.”
Switching among Clouds: An Approach to Ensure Cloud Service Portability – International Journal of Computer Applications Feb ’13

Note: I originally proposed the use of an external delegate storage service in the context of cloud service metering in April ’10 in the article “Metering in the Cloud: Visualizations Part 1 (PDF)”.

“Cloud vendors have a vested interest in making it drop-dead simple and cheap to put your data on their respective clouds….And, they don’t necessarily see the value in making the return trip so easy and that’s what has people spooked….When you move to cloud, you should be increasing your choices, not decreasing them. You don’t buy three on-premises apps but you can use three services from three vendors in the cloud….Bill Gerhardt, director of Cisco Systems’ internet solutions group’s service provider practice, agreed. “We need to sort out data portability. Customers ask: ‘If I give you all this data, how do I retrieve that data if I want to go somewhere else? Many cloud companies don’t have a clear exit route.”
Fear of lock-in dampens cloud adoption – GigaOM Feb ’13

“In 2010 the US analyst firm Gartner created a cloud computing bill of rights, the first and Gartner would argue the most important aspect of this bill was data ownership. While many online services support the Gartner proposal, data ownership, transparency and most importantly portability are not in the interest of the majority of cloud service providers….Conclusion….Despite the well reported problems, Cloud Computing is growing. The number of businesses using online services is increasing at a huge rate. The attraction is easy to understand; the combination of low cost and immediate availability makes a lot of sense. However, millions of businesses are now completely reliant on third party services owned and run by corporates with their own agendas and business models. Are businesses sleep walking into a potential disaster?”
Cloud Computing: Are businesses sleep walking into a potential disaster? – Aug ’12

“Verizon Wireless has begun selling information about its customers’ geographical locations, app usage, and Web browsing activities, a move that raises privacy questions and could brush up against federal wiretapping law…”We’re able to view just everything that they do,” Bill Diggins, U.S. chief for the Verizon Wireless marketing initiative, told an industry conference earlier this year. “And that’s really where data is going today. Data is the new oil.
Verizon draws fire for monitoring app usage, browsing habits – Oct ’12

“When a customer seeks to terminate an outsourcing agreement, the service provider typically denies that any material breach of con- tract has occurred or that a customer has any basis to terminate for cause, and demands payment in full or a large termination fee, representing liquidated damages for lost business. The service provider may simply hold the customer’s data hostage until payment is made…The data hostage and arbitration clauses together may give a breaching service provider the leverage to coerce an outsourcing business to pay a termination fee to which it is not entitled…There are two possible solutions to the data hostage dilemma; the first requires government intervention while the second requires addition of a contract term creating a private expedited dispute resolution mechanism to remove the data from the service provider while arbitration or litigation proceeds…Outsourcing businesses may not recognize the coercive power their service providers stand to gain when data hostage clauses are combined with dispute resolution clauses that permit substantial delays in resolving disputes.”
Walking from Cloud to Cloud: The Portability Issue in Cloud Computing – ASHINGTON JOURNAL OF LAW, TECHNOLOGY & ARTS ’10

“Traveling across international borders can be as tricky for data as it is for a person, even when that information is transferred in the cloud. If you have any doubts, just ask the financial sector….Kristen J. Matthews, head of the privacy and data security group at the law firm Proskauer, explains that since cloud computing transfers personal data outside of the group bound by corporate rules, the BCR (Binding Corporate Rules) is insufficient. But since cloud computing is becoming increasingly popular, the law needs to catch up to the technology.”
Crossing Borders in the Cloud – HPC In the Cloud Mar ’13

In a hybrid, cloud and on-premise world, the issue of knowing where trusted data should/does reside proved a big issue among the CIO respondents, according to Broome. “The ability to avoid SaaS silos and easily obtain consistent data integration and data quality across on-premise and cloud-based data is definitely a top concern among those IT decision makers we surveyed” he said.”
IdevNews – CIOs Bullish on Cloud Benefits, But Worry About SaaS Data Silos

“If startup history tells us anything, it’s that the majority of cloud services launched in the past few years won’t be around forever. The fact that they just vanish into the ether makes the problem quite perplexing.”
How to protect your company against vanishing cloud services – May ’13

“The natural tendency expressed by the Constructal Law (toward easier flow, and greater access to inputs over time) is visible everywhere because all natural flow systems possess freedom. Without freedom to change, design and evolution cannot happen. With freedom, a natural flow system evolves with progressively greater flow performance. Freedom is the sine qua non condition for improvements over time. Freedom is good for design.”
“Freedom Is Good for Design,” How to Use Constructal Theory to Liberate Any Flow System – Forbes – Mar ’12

“Even more troubling, some European activists are calling for data-storage rules to thwart the U.S. government’s surveillance advantage. The best way to keep the American government from snooping is to have foreigners’ data stored locally so that local governments – and not U.S. spy agencies — get to say when and how that data may be used. And that means nations will force U.S.-based Internet giants like Google, Facebook, and Twitter, to store their user data in-country, or will redirect users to domestic businesses that are not so easily bent to the American government’s wishes.”
U.S Government Surveillance: Bad for Silicon Valley, Bad for Democracy Around the World – The Atlantic – June ’13

“Why I bang on about cloud computing is because every organization is now under the cost [pressure] to think about migrating their data to the cloud, and overwhelmingly the cloud computing industry is an American industry,” Bowden said, according to Techworld. And these U.S. cloud service providers are subject to U.S. laws and requests for information under FISA. Critics hold that at least if you run your own operations, chances are you’ll know if you’ve been asked for information but if you use SaaS applications run by Google or Microsoft, your provider could be asked for that data and turn it over without you even knowing.
If PRISM doesn’t freak you out about cloud computing, maybe it should, says privacy expert – Gigaom – June ’13