Going Beyond State/Structure Reflection in Java with Behavioral Tracking
When it comes to behavioral awareness pretty much all application software is stateless (or clueless). Threads will repeatedly execute the same code path in servicing a path even after it has failed for the umpteenth consecutive time. No matter how intelligent software may appear to its users, in assisting them in their tasks, it is inherently dumb (or at least suffering from a severe and prolonged bout of amnesia).
The primarily reason for this is that code has no means to introspect on its execution behavior (and performance) beyond its immediate (current) method and call sites. Code needs to be able to introspect its own execution behavior (including direct/indirect call dependencies) as well as that of its called (methods) and their called methods and so on. In doing so it can define (and redefine) normal execution behavior of a method (and all the methods it calls) and use this to reason about variation in operation, determine the drivers of such variation and predict (immediate) future performance. This goes far beyond primitive software based circuit breakers which are generally applied at remote call exit points. What is the point having a thread in servicing a request pass down all the various layers only then to be (temporarily) blocked or incur an expensive and problematic exception at such a point because of an ongoing fault. If a thread is aware of its behavior (and is able to distinguish one from another) it should be able to automatically short circuit the execution much further up the caller chain (much closer to the entry point). It should be able to look at a method (or a partial call stack) and quickly predict whether execution of the method is going to involve the usage of a faulty component. To do that it needs to be able to ask the runtime what happens when this method executes. The “happens” should include the methods that are likely to be executed and the typical resource consumption in doing so.
Today modern language runtimes such as Java/JVM and C#/.NET offer a means to dynamically introspect (and interact) with objects (and classes) via reflected methods and fields handles but there is no way for the software to see beyond the scope of its code block when execution control is passed (via a call site) to another method.
Whilst a method can at runtime, via a horrendously expensive thread call stack dump, determine its callers it can’t see what it actually does, in particular which methods it calls directly (or indirectly). This is all left on the drawing board during development – well actually within the source code (and editor). Self regulated software, which we believe is the only viable foundational approach to addressing IT management complexity, needs to be able to answer fundamental questions like “what did I (the thread) just do (call/consume) between two points in time or in executing this called method”.
Note: Networks have supported a slightly similar mechanism via path trace routing and many operational aspects of supply chain management rely on similar means of inspection both of which are focused on the interconnectedness of (service) delivery. The application is the new edge network which is why we have invested heavily in bringing QoS up the stack.
To make this less abstract lets see how to go about transforming a simple class into one that can reflect on its execution behavior.
Lets now make the
main() method able to reflect on its execution behavior between the
Running the above with JXInsight/OpenCore dynamic bytecode instrumentation agent results in the following output.
We can go much further with behavioral reflection by reporting on actual consumption of resources which in the Probes Open API are represented as meters.
Running the revised code results in the following output.
Now just imagine what the impact would be on library development if such capabilities were fully supported in application runtimes. We believe this would lead to exciting new library design and the re-development of many existing libraries eventually resulting in truly adaptive, self stabilizing, self regulated software applications and services.
Naturally there is a marginal cost to such collection. But this for the most part can be addressed using intelligent activity metering eliminating much of the noise and waste from measurement.
Incidentally you can use the above Open API code (within a
ProbesInterceptor) with our amazing new discrete event simulation solution, JXInsight/Simz, to introspect the behavior of methods across an entire distributed infrastructure looking for deviations across nodes and runtimes which might be indicative of differences in code, environment and workload.
I presented a similar vision to the Paris Java User Group this week and the response was extremely encouraging. Many of those that watched the above being demonstrated live came up to me afterwards to discuss the possibilities for this technology and its potential.