Visit Azul.com Support

Using Java Flight Recorder

The Java Flight Recorder (JFR) feature allows you to record JVM, system, and Java level events to monitor the behavior and troubleshoot problems in your Java applications. Keeping the overhead of the recording process low, Azul Zing Builds of OpenJDK (Zing) Java Flight Recorder records a large amount of information about your system from command-line parameters and system properties to time spent executing a piece of code.

JFR is available in Zing without the need for command-line options to enable it.

To collect a JFR recording for your application, you can either use the jcmd tool, interactively connect to a Java process from the Azul Mission Control desktop application, or add -XX:StartFlightRecording to the application’s java command line to collect the recording from JVM startup.

Azul Mission Control is recommended for browsing JFR recordings generated by Azul Prime. This tool is available for download here:

https://www.azul.com/products/components/azul-mission-control/.

Collect a JFR Recording with jcmd

On the same host and userid as the java process under analysis, run the following command to start a recording:

 
jcmd JVMID JFR.start filename=rec.jfr

Where JVMID is either the process ID or the classname assigned with the java process as shown when running jcmd without parameters.

To stop a recording, use:

 
jcmd JVMID JFR.stop name=1

To start a recording with more detailed metrics, use the following instead:

 
jcmd JVMID JFR.start filename=rec.jfr settings=profile

See the JFR options table below for a comprehensive list of options.

Collect a JFR Recording using Azul Mission Control

You can directly collect the JFR in Azul mission Control without using the command line or terminal. Collecting a JFR recording in Azul Mission Control requires establishing a connection between Azul Mission Control and the running JVM.

Azul Mission Control allows either local or remote connections to the JVM. Remote connections require enabling the JVM’s JMX connector first. More details can be found in the Azul Mission Control documentation, Connecting to a Remote JVM.

After the connection is established, you can start collecting the JFR using the steps outlined in the Azul Mission Control documentation, Starting a JFR Recording in Azul Mission Control.

Collect a JFR Recording from JVM Startup

For detailed performance analysis, you might prefer to start a JFR recording directly when launching a java process.

To start recording from JVM startup, add -XX:StartFlightRecording to the application’s java command line.

Use a comma "," for separation when using multiple options. For example:

 
java -XX:StartFlightRecording=duration=30s,filename=myrecording.jfr TestRecorder

See the JFR options table below for a comprehensive list of options.

Analyzing a JFR Recording

It is recommended to use Azul Mission Control to view JFR recordings.

Alternatively, a Java Flight recording can be viewed using the jfr command-line option.

Using Azul Mission Control for JFR Analysis

In Azul Mission Control, You can either drag and drop a JFR recording from any host into Azul Mission Control, or directly collect the JFR using Azul Mission Control.

Azul Mission Control has more uses than just recording and viewing JFR recordings, such as actively monitoring JVM resource usage through MBeans. More info about Azul Mission Control can be found in the Azul Mission Control Documentation.

Using jfr Command Line Option for JFR Analysis

If you prefer to analyze JFR recordings from the command line or terminal, you can use the jfr option.

Example usage of the jfr command:

 
$JAVA_HOME/bin/jfr print rec.jfr > file.txt

Using Zing Live Objects

Zing Live Objects help determine the type of objects that are leaking while troubleshooting memory leaks. To identify and address a memory leak using information gathered in a Zing flight recording, complete the following steps.

To create a Zing flight recording for inspection, use the following arguments:

 
java -XX:StartFlightRecording=duration=<recording_time_duration>,settings=profile,filename=<recording_filename>.jfr

The Flight Recorder has two configuration files: default and profile.

To enable the Live Objects feature, you need to enable ZingOldObjectSample in the configuration file. By default, ZingOldObjectSample is enabled only in the profile configuration file in versions of Zing prior to 19.12.0.0. Starting with ZVM 19.12.0.0, ZingOldObjectEvent is disabled by default in all configurations.

Note
  • The JVMInformation, GarbageCollection, and OldGarbageCollection events must be enabled in your configuration file in order to analyze Live Objects in Azul Mission Control. The listed events are enabled in the default and profile configuration files by default.

  • The profile settings may increase performance overhead due to collecting more data by the Java Flight Recorder. Customizing any settings file, either default or profile, may incur increased performance overhead.

By default, the amount of live set events recorded by Java Flight Recorder is set to 99%. To decrease the recording file size, you can set the value of the JFRDescribeLiveSetPercentage command-line option to a number less than 99:

 
java -XX:JFRDescribeLiveSetPercentage=90 -XX:StartFlightRecording=duration=<recording_time_duration>,settings=profile,filename=<recording_filename>.jfr

When a recording is started, the output should look similar to the snippet below:

 
$ /home/username/<JDK_installation_path>/bin/java -XX:StartFlightRecording=settings=profile,filename=myrecording.jfr Started recording 1. Use jcmd 32747 JFR.dump name=1 to copy recording data to file.

If just a filename without a path is specified (i.e., filename=<recording_filename>.jfr), the flight recording is saved to the directory from which you ran the command.

To specify the path of your choice, use:

 
filename=<path>/<recording_filename>.jfr

Once you create a Zing Flight recording, start the Azul Mission Control tool and open your Zing Flight recording from the File menu.

If you do not have Azul Mission Control on your machine, download the installer file from Download Azul Mission Control and see Downloading and Starting Azul Mission Control for installation instructions if needed.

In the Outline navigation tree, select Memory > Zing Live Objects to display the set of live objects captured in the recording.

Study and compare GC cycles in the table with garbage collection identifiers (GC ID) for a live set suddenly increasing in size which indicates a memory leak.

Live objects types marked during each selected GC cycle display in the table to the right when a specific GC cycle is chosen.

compare gc cycles

Sort out live objects in a GC cycle by the total size to track the live object type with the biggest size and select a potentially leaking object type in the Referent column.

All items that refer to a potentially leaking object type display in the Referrer column in the bottom table. The latter helps identify the source of the leaking object by reference count.

(Optional) To retrieve more information, repeat step 5 if needed.

When a potentially leaking class is identified on the Zing Live Objects sub-tab, use the Memory tab to get information about the allocation site. The Memory tab contains allocation sites, which lead to allocation in a new thread local allocation buffer (TLAB) or allocation outside a TLAB, sorted by type. Checking stack traces for a potentially leaking class can help to stop the real place in code where the leak occurs.

In case the Memory tab does not help to find the leak, but the leak is still suspected, the Method Profiling tab might be used for this purpose. It is less convenient compared to the Memory tab, as it shows method calls of concrete types, not allocation sites.

Usually, the Zing Live Objects sub-tab and the Memory tab are all you need to find a leak.

Inspect the Stack Trace of your candidates to track where most memory allocation pressure comes from.

Using TTSP JFR Events

Azul Platform Prime 20.07.0.0 introduces support for profiling Time To Safepoint (TTSP) issues based on data about Zing events collected in a Java Flight Recording (JFR).

To profile an existing TTSP issue using information gathered in a flight recording, complete the following steps:

  1. Create a flight recording for inspection using the following arguments:

     
    java -XX:StartFlightRecording=name=<recording_name>,filename=<recording_filename>.jfr,settings=profile

    The Flight Recorder has default and profile configuration files. By default, the TimeToSafepointIssue event, which activates TTSP issue profiling, is enabled in all configurations. Use the following options to configure the event settings file for the Java Flight Recorder:

    Configuring Option Description Default

    -XX:[+/-]SafepointWaitTimeProfiler

    Generates asynchronous signals to log safepoint issues.

    true

    -XX:SafepointProfilerRealTimeSignalNumber=<number>

    Specifies the number of real time signals used for safepoint profiling.

    4

    -XX:SafepointProfilerThresholdMS=<number>

    Specifies the number of milliseconds to start safepoint profiling.

    0

    -XX:SafepointProfilerThreshold=<number>

    Specifies the threshold percentage value to start safepoint profiling.

    2

    -XX:SafepointProfilerInterval=<number>

    Specifies the interval percentage value for safepoint profiling.

    10

    -XX:SafepointProfilerMaxBuffers=<number>

    Specifies the number of safepoint profiler buffers to track.

    100

    -XX:SafepointWaitTimeProfilerLog=<log file>

    Specifies the log file to write safepoint profiling data to.

    stderr

    Note
    The profile settings may increase performance overhead due to collecting more data by the Java Flight Recorder. Customizing any settings file, either default or profile, may incur increased performance overhead.
  2. Once you create a Zing Flight recording, start the Azul Mission Control tool and open your Zing Flight recording from the File menu.

    If you do not have Azul Mission Control on your machine, download the installer file from Download Azul Mission Control and see the Installation Guidelines if needed.

  3. In the Outline navigation tree, choose Event Browser.

  4. In the Event Types Tree, select Java Virtual Machine > Runtime > Safepoint to display the set of Safepoint event types captured in the recording.

    ttsp issue
  5. In case of a TTSP issue, check the Event Browser table.

    The contents of the Stack Trace window may help you understand the cause of the issue.

JFR Options

Option Description Default

settings=<path/to/settings/file>

Specifies the configuration file to be used. In case of a custom configuration file, use the full path to the .jfc file. Otherwise, use one of the following presets:

  • default - Collects less data than profile but is more appropriate for runs where the JFR recording is continuously running side by side with the application.

  • profile - Collects more data than default but adds more performance overhead.

default

disk=<true, false>

Controls whether to create a temporary recording file under the path set in the repository option. Temporary recording means that a file is written to the system temporary directory during the time of the recording and is deleted afterwards. This recording file is populated gradually, unlike the file generated using the filename option, which is only populated once the recording finishes.

true

repository=<path/to/temp/recording>

Specifies the path for a temporary recording if the disk option is enabled.

/tmp/

filename=<filename.jfr>

Specifies the file name of the generated recording. You can add a path to the recording filename to change the file location. The recording file stays empty and only gets populated once the recording finishes.

hotspot-pid.jfr

duration=<time in specified unit>

Specifies the recording length. Used time units: ns/ms/s/m/h/d. For example, 1800s or 30m.

0, unlimited

delay=<time in specified unit>

Specifies the amount of time that the recorder must wait before starting to record. Used time units: ns/ms/s/m/h/d. For example, 1800s or 30m.

0

dumponexit=<true, false>

Controls whether to generate a recording file (dump a recording) if the JVM shuts down and the filename option has not been specified. The dumped recording filename is hotspot-pid.jfr

false

name=<identifier>

Specifies a name identifier for the recording. This is useful when working from jcmd and operating multiple recordings running at the same time.

1,2,3,4,…​