Skip to content

JXInsight/OpenCore 6.4.EA.1 Released – Adaptive “Throughput” Control

In the first early access build of JXInsight/OpenCore 6.4 “ACE” we have added a new metering extension, tvalve, that self regulates the concurrency of probes associated with a valve for the purpose of throughput optimization using various flavors of local search (optimization) including hill climbing, reactive search optimization (RSO) and tabu search.

Local search is a meta-heuristic method for solving computationally hard optimization problems. Local search can be used on problems that can be formulated as finding a solution maximizing a criterion among a number of candidate solutions. Local search algorithms move from solution to solution in the space of candidate solutions (the search space) by applying local changes, until a solution deemed optimal is found or a time bound is elapsed. – Wikipedia

We have added numerous control set points to the tvalve probes provider to allow fine tuning of nearly every adaptive aspect of its operation but for the most part one need only specify the following system properties.

jxinsight.server.probes.tvalve.enabled=true
jxinsight.server.probes.tvalves=${v},${v},...
jxinsight.server.probes.tvalve.${v}.name.groups=...

To test the adaptiveness and effectiveness of the search optimization I’ve created 3 micro benchmarks that are greatly impacted by the degree of concurrency – atomic counter increment, object allocation (leading to gc pauses), and monitor contention. Each benchmark was run twice within a single process execution with a total of 20 threads each performing 20,000 methods invocations of a call() method from within a run() method.

For each of the test cases below I am going to use the following jxinsight.override.config file contents which injects a “throughput” valve into the call() method by way of an interception of the metering instrumentation weaved into the class at load-time by our JVM agent.

j.s.p.tvalve.enabled=true
j.s.p.tvalves=v
j.s.p.tvalve.v.name.groups=samples.A.call

Here is the relevant code snippet for the atomic counter increment test case.

Running the test case without any instrumentation results in a benchmark time of 104 seconds for each innovation of the run() method. Running the test with JXInsight/OpenCore as specified above results in a clock time of 56 seconds in the first invocation dropping down to 50 seconds in the second innovation due to narrowing of the search space. This is a huge savings in time and a significant gain in throughput all achieved whilst metering (profiling) the code as well as controlling the execution.

Below is the code used to simulate the typical high object allocation rates found in web application request processing.

Here again our adaptive control of concurrency optimizes the throughput bringing down the overall wall clock time for the fixed work volume from 145 seconds to just under 106 seconds.

The following code creates a very high degree of monitor contention via a quick succession of lock acquisition and release.

Whilst this was the fastest of all benchmark tests the overall wall clock time did still drop from 36 seconds, without any instrumentation or measurement, down to 21 seconds with activity resource metering and adaptive control via our user defined “throughput” valve.

To understand what exactly is happening underneath in applying adaptive control it is best to look at the throughput (calls/s) for samples.A.call() method over the course of a benchmark with our metrics metering extension enabled.

Here are the results for the atomic counter increment test case. Notice the throughput dips in the overview chart and the lengthening of the space between the dips. This is the adaptive control probing the limits of its concurrency and backtracking on such when the gain is negligible or negative.

The throughput for the highly contended monitor test case has a similar pattern. As it allows additional workload to pass through the valve it observes a drop (dip) in the overall throughput then re-adjusts resulting in the throughput returning to a previous local high of approximately 20,000 calls a second.

Without the controller positively influencing (in line with its goal) the execution behavior of the metered software the throughput remains stable at just over 10,000 a second which is half of what is achieved with adaptive control.

Further information on the large number of control set points offered by the tvalve probes provider metering extension will be published shortly on our developer site.

JXInsight/OpenCore 6.4.EA.2 Released – Adaptive “Response (Time)” Control