An important aspect of all JXInsight trace extensions is performance data collection - the amount captured and its structure. JXInsight is one of the most configurable J2EE™ Application Performance Management (APM) solutions on the market. All extensions have been designed to give software performance engineers and application management staff as much flexibility in defining configurations appropriate to each individual application and component. R&D staff at JInspired, who have years of experience in building J2EE applications and containers, recognized that it would not be possible to offer an universal APM solution unless we made configurability and extensibility are integral elements of the overall JXInsight product design. In this article I will discuss the first element using examples based on JXInsight's Servlet / JSP trace extension.

IMPORTANT JXINSIGHT 4.0 CHANGE NOTE In JXInsight 4.0 the ServletFilter class has been renamed. Please make the following configuration configuration changes. RENAME com.jinspired.jdbinsight.trace.ext.servlet.TraceFilter TO com.jinspired.jxinsight.trace.ext.servlet.TraceFilter.

Problem
Most performance management solutions group all Servlet/JSP requests having the same base URL into a single traced request with clock time statistics aggregated. In some cases it is the actual Servlet/JSP class that is profiled with clock time statistics aggregated at the class.method level. This presents a problem to users of such solutions because there are many J2EE™ web applications which have a single javax.servlet.Servlet class processing all requests where the servlet uses the request parameters to determine the actual request handler to forward the request to.


Solution
JXInsight allows J2EE™ application management staff as much flexibility in specifying the following:

Which requests to trace? What request data is captured? How are profile statistics aggregated?

JXInsight's trace extension uses the standard mechanism within the J2EE™ platform that has tooling support in all Java IDE's and Application Servers administration consoles - javax.servlet.ServletFilter. JXInsight provides an ServletFilter implementation that traces and profiles web application requests.


Which requests to trace?
To trace and profile web application requests first define the filter in the applications web.xml. Note: Apache Tomcat supports the introduction of servlet filters across all deployed web application with no application config changes.

<filter>  <filter-name>jxinsight</filter-name>   <filter-class>com.jinspired.jdbinsight.trace.ext.servlet.TraceFilter</filter-class> </filter>

The following applies the JXInsight filter to all JSP requests.

<filter-mapping>   <filter-name>jxinsight</filter-name>   <url-pattern>*.jsp</url-pattern> </filter-mapping>

To trace and profile all web application requests.

<filter-mapping>   <filter-name>jxinsight</filter-name>   <url-pattern>/*</url-pattern> </filter-mapping>

What request data is captured?
The JXInsight servlet filter can be configured to delimit a request into distinct trace identifiers using the trace.identifiers initialization parameter. The following specification separates a request provided to the filter into protocol (HTTP, HTTPS), method (POST, GET, or PUT), and URL.

<filter>   <filter-name>jxinsight</filter-name>   <filter-class>com.jinspired.jdbinsight.trace.ext.servlet.TraceFilter</filter-class>   <init-param>     <param-name>trace.identifiers</param-name>     <param-value>${servlet.protocol},${servlet.method},${servlet.url}</param-value>   </init-param> </filter>

The list of possible variables obtained from either the request, servlet or web application are:

${servlet.protocol}
${servlet.remotehost}
${servlet.remoteaddress}
${servlet.method}
${servlet.port}
${servlet.querystring}
${servlet.sessionid}
${trace.timestamp}
${servlet.url}
${servlet.uri}
${servlet.contextpath}
${servlet.pathinfo}
${servlet.servletpath}
${servlet.user}
${servlet.authtype}

Variables must be delimited by a comma. It is also possible to embed constant strings.

It is possible to define multiple servlet filters with the same JXInsight servlet filter class name but different init parameters which can then be mapped to different URL patterns. This allows for different request calling mechanisms to be supported within the same application and web context.


How are profile statistics aggregated?
There are 2 types of traces in JXInsight. The first type of trace is added and removed from a thread's trace stack for grouping of child trace statistics as well as delimiting traces. This type does not have a recorded trace point (Java call stack) and its statistics are derived solely from it children. By default all top level traces started within a monitored JVM have a grouping on host and process. This is specified by the system property: -Djdbinsight.server.tracer.root=${host},${process}

The second type of trace is the actual execution point within the JVM. For this type JXInsight records the Java call stack as well as start and stop counters for clock time, cpu time, object allocation, thread blocking, thread waiting, and GC time. When a trace.identifiers parameter value consists of more than one identifier then all identifiers (variables) preceding the last are of type 1. The last trace identifier is always of type 2. In the servlet example above the protocol and method identifiers are of type 1 and the URL of type 2. Within the JXInsight console type 1 and type 2 are visually distinguished by a empty gray box graphic and a gray box with containing an arrow graphic respectively.




Sample Web Application
Java™ Adventure Builder is a fictional sample application from the Sun Java BluePrints for the Enterprise. This web application employs 2 servlet controllers that forward requests on to other web components based on url request patterns (*.do or *.screen). In the case of the Cart tabbed view shown below it is the URL + the request parameter, tab, which dictates the actual tabbed displayed.




Standard Configuration
The following XML was inserted in the Adventures application standard web.xml file. By default the JXInsight servlet filter will delimit the trace into protocol and URL. This configuration is typical of what is captured in other APM solutions.



The JXInsight console screen shot below shows the profile trace tree constructed with the above configuration. The host and process aggregate traces have been appended to the single top level trace - ${servlet.protocol}. The servlet protocol trace identifier aggregates all child traces which in this case is ${servlet.url}. The statistics for an aggregate trace are displayed in columns with the '(S)' suffix . The selected table row is the actual trace execution point. The problem with our analysis of the profile snapshot is that it is not possible break down hits on the cart page by each of the 5 tabs.




Revised Configuration
A quick and easy solution to the problem above would be to change the servlet trace identifiers via the 'trace.identifiers' init parameter. Note: Ideally I should have created a new filter and filter mapping applicable to requests on the Cart page.



The following screen shot shows the profile trace tree constructed with the revised configuration. It is now possible to see clock times and memory allocation (in bytes) per tab while still having (aggregated) statistics at the URL level.




Grouping & Ordering
The ability to order the grouping of traces is a powerful and unique feature of JXInsight trace extensions. Developers, testers, architects, and system administrators can change the ordering of traces identifiers to better reflect the execution nature of each web application and component while aligning to the deployment model under management. In the following configuration the context path has replaced the full URL to enhance readability. The HTTP method has been added to distinguish upload operations.



The following screen shot shows the profile trace tree constructed with the above configuration.



In the following configuration the HTTP method has been placed in front of the context path to allow for aggregation at the HTTP method type as well as trace separation.



The following screen shot shows the profile trace tree constructed with the above configuration.




Transaction Paths
Because JDBInsight transaction paths are identified by the Java call stack, trace stack and SQL context of contained visits (steps) any configuration change at the servlet filter level impacts the number of uniquely identified transaction paths (patterns) resulting from HTTP request processing. This is extremely useful allowing for flexibility granularity of aggregated statistics for JDBC™ transactions. The screen shot below shows the grouping of transaction paths (child tree nodes) based on the above configuration.




Trace Roots and Constants
All top level traces can be prefixed with global trace identifiers. This is specified by the JVM system property:
-Djdbinsight.server.tracer.root=${host},${process}

Possible variables include:

${host}
${process}
${threadgroup}
${thread}

Once again this flexibility is extremely useful in creating software performance execution models that are aligned to how a particular customer wishes to managed and monitor the performance and reliability of deployed applications. With the ability to embed string constants in the root trace identifiers one can group and aggregate trace and transaction path statistics based on customer specified identifiers such as a cluster, owning department, or internal project name.

The screen shot below shows the output produced when the system property is specified as
-Djdbinsight.server.tracer.root=sun-jsas-cluster,adventure-app,${threadgroup},${thread}



If this configuration is used consistently across a set distributed application servers then it is relatively easy to derive a global application level performance measurement by merging snapshots within the JXInsight console.


Summary
In this article I have touched briefly on the configurability of JXnsight's Servlet/JSP trace extension. There are still many configuration options and application scenarios to be discussed but I would hope that with the information provided, formulating a suitable application management solution will be less of a challenge than it is today with similar products.


William Louth, JXInsight's Product Architect