Coordinated Restore at Checkpoint Usage Guidelines

Table of Contents

Using CRaC in Your Application
- Adding the CRaC API
- Implementing the CRaC Resource
Running an Application With CRaC
Example Code

Need help?

Schedule a consultation with an Azul performance expert.

Your application may be ready for CRaC out-of-the-box! The simplest way is trying to generate a checkpoint. When you meet a CheckpointException, (that describes the problematic state in the application), or if you want to have more control, check the "Implementing the CRaC Resource" section.

Using CRaC in Your Application

Adding the CRaC API

The library provided by org.crac is designed to provide a smooth CRaC adoption. Add this library to build an application that uses the CRaC API, to be able to run it on Java runtimes with CRaC, or without any implementation.

You can find the library in the Maven Repository.

Maven

 <dependency>
  <groupId>org.crac</groupId>
  <artifactId>crac</artifactId>
  <version>${crac.version}</version>
</dependency>

Gradle

 implementation 'org.crac:crac:1.3.0'

Functionality

During runtime, org.crac uses reflection to detect the CRaC implementation. When available, all requests to org.crac are passed to the implementation. Otherwise, requests are forwarded to a dummy implementation.

The dummy implementation allows an application to run but not to use CRaC:

Resources can be registered for notification.
Checkpoint request fails with an exception.

Implementing the CRaC Resource

To use the API, you need to identify all classes in your code that are considered "resources": classes that must be notified when a checkpoint is about to be made and when a restore has happened. The API provides an eponymous interface, Resource, which must be implemented for the identified classes. There are only two methods, beforeCheckpoint() and afterRestore() which are used as callbacks by the JVM.

The CRaC JavaDoc is available here.

Example use case:

If a class reads configuration from a file, the file must be closed in the beforeCheckpoint() method.
In the afterRestore() method, the file can be opened again to check configuration updates.

The same applies to network connections, and you can also use the methods to deal with a sudden change in the system clock, which might impact things like cache timeouts.

All Resources in the application must be registered with the JVM, which can be achieved by obtaining a CRaC Context and using the register() method. Although you can create your own Context, the simplest way is to use the global Context obtained via the Core class’s static getGlobalContext() method.

It’s important to register the Resources in the right order because this order is used to call the beforeCheckpoint methods. However, the afterRestore methods are called in the opposite order. This approach simplifies things if there is a particular sequence in which things need to be prepared for a checkpoint; when restoring, there is a predictable inverse sequence.

Running an Application With CRaC

Currently, the CRaC functionality is only available on Linux/x64 and Linux/Arm64, in version 17 of Azul Zulu Builds of OpenJDK. This means, for now, you can run an application with CRaC on any system thanks to the crac.org dependency, but only on the specified OS systems the CRaC functionality in the JVM will work.

Note	Some VMs, like Parallels, cannot run foreign CPU instructions, and you need a build matching your CPU. Some other virtualization environments, like WSL, do not provide a complete feature set of Linux kernel, limiting the CRaC functionality in the current version.

Running CRaC on a Linux System

Selecting a Runtime

Currently, CRaC is only available on Linux/x64 and Linux/Arm64, in version 17 of Azul Zulu Builds of OpenJDK, as of the release of April 2023. Only the bundles with -crac- in the name, have the integrated CRaC functionality.

On the download section of the Azul website, you can use the "Java Package" > "JDK CRaC" filter.

Note	The JDK archive should be extracted with sudo.

 $ sudo tar zxf <jdk>.tar.gz

Generating a Checkpoint

Start the JVM with an additional flag -XX:CRaCCheckpointTo so it’s prepared to create a checkpoint:

 java -XX:CRaCCheckpointTo=$HOME/crac-image/ -jar my_app.jar

Note	This eventually generates a set of files in the given directory, which cumulative size will be roughly the size of the JVM resident memory.

Then you can use two ways to trigger the checkpoint:

From outside the JVM, using jcmd with JDK.checkpoint.
Programmatically within your application, by adding a call to Core.checkpointRestore() in the flow of your program, where you want the checkpoint to be created. The method will return when the restore has been completed.

Using a Checkpoint

When a checkpoint is available, you can (re)start your application and restore from the image:

 java -XX:CRaCRestoreFrom=$HOME/crac-files/

Running CRaC in a Virtualized Environment (Docker)

Creating a Docker Image

You need an application in a runnable JAR file.

Create a Dockerfile, based on the following minimum file:

 FROM ubuntu:20.04

RUN apt-get update -y

ADD "https://cdn.azul.com/zulu/bin/zulu17.44.17-ca-crac-jdk17.0.8-linux_x64.tar.gz" /opt/
RUN cd /opt/ && tar -xzf zulu17.42.21-ca-crac-jdk17.0.7-linux_x64.tar.gz && rm zulu17.42.21-ca-crac-jdk17.0.7-linux_x64.tar.gz

ENV JAVA_HOME /opt/zulu17.42.21-ca-crac-jdk17.0.7-linux_x64
ENV PATH $JAVA_HOME/bin:$PATH

COPY build/libs/my_app.jar /opt/app/my_app.jar

Build the Docker image with:
```
 docker build -t my_app_on_crac .
```

Starting an Application in a Docker Container

Run a Docker container with:

 docker run -it --cap-add=CHECKPOINT_RESTORE --cap-add=SYS_PTRACE --rm --name my_app_on_crac \
    -v $PWD/crac-files:/opt/crac-files my_app_on_crac \
    bash -c '( echo 128 > /proc/sys/kernel/ns_last_pid ) 2>/dev/null || while [ $(cat /proc/sys/kernel/ns_last_pid) -lt 128 ]; do :; done;
    java -XX:CRaCCheckpointTo=/opt/crac-files -jar /opt/app/my_app.jar'

Note	The write to `ns_last_pid` is required for the latter unprivileged restore.

Leave the shell window open and the application running.

Creating the Checkpoint

Open another shell window.
In this window run:
```
 docker exec my_app_on_crac jcmd PID-OR-NAME JDK.checkpoint
```
Note
You can find PID or NAME to provide to jcmd by executing just jcmd in the container.
If everything is ok, you see that in the first shell window the checkpoint was created and your application was closed.

Creating a Docker Image with Checkpoint

Create a Dockerfile with the checkpoint:

 FROM my_app_on_crac
COPY crac-files /opt/

Build the Docker image with

 docker build -t my_app_on_crac_restore .

Run the Docker Container From the Checkpoint

Run:

 docker run -it --rm my_app_on_crac_restore java \
    -XX:CRaCRestoreFrom=/opt/crac-files

Your application now starts much faster from the saved checkpoint.

Note	You can run the Docker container also on macOS or Windows, as long as the machine you are running it on has a x64 cpu architecture (Intel/AMD).

Running CRaC on Windows or macOS

Make sure to use an x64 system. ARM-systems is not supported at this moment.
Run your application in a virtualized Linux system, e.g. following the Docker approach described above. For instance, on macOS you can use Docker, Podman, Parallels, VirtualBox,…

Example Code

Within the CRaC project on GitHub, a fully documented "Step-by-step CRaC support for a Jetty app" is provided.