Skip to content

Simulation and Time Synchronization of Application Memories

“Learning is experience. Everything else is just information.” -Albert Einstein

Experiencing is Learning
Experiential learning is the process of making meaning from direct experience, i.e., ”learning from experience”. The experience can be staged or left open. The process involves a direct encounter (experience) with a phenomena of interest rather than merely thinking about such, or only considering the possibility of doing something about it.

jxinsight.opencore.timesync. experiential.learning

Reflection is a crucial part of the process of learning. It is an iterative process as learning builds and develops with further reflection and experience. In reflecting on an experience 5 key questions can be asked to improve understanding:

  • Did I notice (observe and recognize) what happened?
  • Why did that happen (and in what manner and form)?
  • Does that happen (often and does it matter)?
  • Why does that happen (beyond the particular event experienced)?
  • How can I use that (to alter my behavior in the future)?

These questions should be proposed and answered in attempting to understand application performance problems. But how can we experience the past and in such a way that it will feel real and to some degree uncertain at each point in time (in order to tune our intuition). Here is were simulated playback of past application memories can be a crucial part of learning to manage and operate complex software systems.

Experiencing The Past with Playback Simulations
In JXInsight/OpenCore 6.4.4 we have released a new metering extension, timesync, that controls the execution of the metering runtime by ensuring the time differences between two events reflect the difference in elapse time. This difference typically occurs when a metering recording is played back at full speed, without any delay other than the time incurred in changing the meters to reflect the recorded readings as well the metering of the probe itself. This aspect of playback has been covered in How to Execute Software Behavior Faster than (Wall Clock) Time.

The timesync metering extension is similar to the plugin introduced in The Power of Slow in Application Performance Analysis — Record & Playback, but far more efficient and scalable as well as being built directly into the metering runtime. The extension allows us to observe events in the past with the same degree of passage time as if we were actually observing the event occur in the present. This capability is extremely useful for IT operations. Instead of simply looking at a static timeline chart or log listing we can train ourselves to recognize events unfold, anticipate what is likely to follow and then more importantly check whether we recognize and anticipate correctly. This is flight simulation training for IT operations and more as we can go back in time and experience the whole timeline sequence again and again as well as pause time itself. We can step through time and assess each point without necessarily having any clues (or knowledge) of what comes next (in the replay), which is invariably the case with post-mortem management reports that visualize the whole monitoring timeline data on a single static chart. Knowing what is coming next changes our view of data points and actions performed. It limits (or biasses) our learning. Would you read a novel after reading the last few pages and then fully appreciate the experience it aims to give. No. Well that is exactly what many post-mortem analysis reports offer.

To enable the timesync extension simply add the following to the jxinsight.override.config used by the tools/opencore-playback.jar library.

jxinsight.server.probes.timesync.enabled=true

Seeing is Believing
A screen recording has been created that demonstrates the power of playback time synchronization. In the video a DataStax (Apache) Cassandra server is started and then a couple times an “insert” benchmark test is run against the instance. During this time the metering engine within the Cassandra runtime is in recording mode, creating a file named probes.log. It is this binary metering recording log that is then played back later when the Cassandra runtime is shutdown. What is important to note during the video is that the OpenCore console, which connects automatically to both runtimes, is completely oblivious to the fact that one of the runtimes is in fact a simulation engine of a past application memory. This runtime looks and acts just like the Cassandra runtime. It has the same number of threads, each doing what they previously did except for not actually do the table inserts execution code but simply mimicking the execution flow by firing the instrumentation probes in order and in context to what was previously fired in the real runtime. A way to think of this is that every method that was instrumented and metered in the real runtime has a no-op implementation in the simulated runtime except for calling other methods that are also no-op implementations and so on.

jxinsight.opencore.timesync.cassandra.diagram

The screen video recording is also available on YouTube and ideally should be viewed in full screen HD mode.