CPU Features

Table of Contents

Generic Target
Specifying CPU Features
Merging CPU Features
Command Line Options

Need help?

Schedule a consultation with an Azul performance expert.

Note	CPU Features are available since `zulu17.46`, `zulu21.30` (October 2023).

CPU features differ from host to host because newer CPUs support newer instructions, and some optional features may disappear over time. The JVM (Java Virtual Machine) depends on CPU features (such as AVX512, SSE3, and others) available on the computer where it is running. The more CPU features the JVM can use, the better performance it can provide. The JVM normally uses all CPU features it detects when it starts.

CRaC cannot reconfigure an already running JVM to stop using CPU features when it is restored on a CPU that does not support them.

Generic Target

For the broadest compatibility target, specify -XX:CPUFeatures=generic during checkpoint. The VM then uses only features available on every x86-64 CPU.

Note	The `generic` target prevents the JVM from using some otherwise available CPU features, so it can reduce performance.

To avoid the performance impact associated with generic, you can follow the advice printed by the VM and specify an exact feature set as detected on the target machine.

Specifying CPU Features

You may create a snapshot (-XX:CRaCCheckpointTo=PATH) on a newer computer and restore it (-XX:CRaCRestoreFrom=PATH) on an older computer.

In that case, you may see a variant of this error message during restore:

 $JAVA_HOME/bin/java -XX:CRaCRestoreFrom=cr
You have to specify -XX:CPUFeatures=0x21421801fcfbd7,0x3e6 together with -XX:CRaCCheckpointTo when making a checkpoint file; specified -XX:CRaCRestoreFrom file contains CPU features 0x4ff7fff9dfcfbf7,0x3e6; missing features of this CPU are 0x4de3de79c000020,0x0 = 3dnowpref, adx, avx512f, avx512dq, avx512cd, avx512bw, avx512vl, sha, avx512_vpopcntdq, avx512_vpclmulqdq, avx512_vaes, avx512_vnni, clflushopt, clwb, avx512_vbmi2, avx512_vbmi, rdpid, fsrm, gfni, avx512_bitalg, pku, ospke, avx512_ifma
If you are sure it will not crash you can override this check by -XX:+UnlockExperimentalVMOptions -XX:+IgnoreCPUFeatures .
<JVM exits here>

Follow the advice printed on the system running -XX:CRaCRestoreFrom=PATH when configuring the system that creates the snapshot (-XX:CRaCCheckpointTo=PATH).

 $JAVA_HOME/bin/java -XX:CRaCCheckpointTo=cr \
    -XX:CPUFeatures=0x21421801fcfbd7,0x3e6 \
    -jar target/spring-boot-0.0.1-SNAPSHOT.jar

Merging CPU Features

If you restore the CRaC image on computers with different CPU features, you may need to use the bitwise AND (&) of the suggested -XX:CPUFeatures values.

 # Computer A used for an image restore:
You have to specify -XX:CPUFeatures=0x21421801fcfbd7,0x3e6 together with -XX:CRaCCheckpointTo when making a checkpoint file; [...]

# Computer B used for an image restore:
You have to specify -XX:CPUFeatures=0x4b03c643c9869,0x173 together with -XX:CRaCCheckpointTo when making a checkpoint file; [...]

# Compute common minimal set of features of computers A and B:
python -c 'print(hex(0x21421801fcfbd7 & 0x4b03c643c9869)+","+hex(0x3e6 & 0x173));'
0x18003c9841,0x162

# Computer used for the image snapshot:
$JAVA_HOME/bin/java -XX:CRaCCheckpointTo=cr -XX:CPUFeatures=0x18003c9841,0x162 -jar target/spring-boot-0.0.1-SNAPSHOT.jar

Command Line Options

-XX:CPUFeatures=<value>

Controls which CPU features the JVM may use and which feature set it records in a CRaC image for restore-time compatibility checks. You can use this option to make an image checkpointed on one CPU restorable on a different CPU. Available values:

native: Use the CPU features available on the current CPU. This is the default.
generic: Use a conservative CPU feature baseline intended to work on all currently available CPUs. If the CRaC image is restored on computers with different CPU features, the simplest solution is to run Java with:
```
 $JAVA_HOME/bin/java -XX:CRaCCheckpointTo=cr \
    -XX:CPUFeatures=generic \
    -jar target/spring-boot-0.0.1-SNAPSHOT.jar
```
This mode can reduce performance because advanced instructions may be disabled. To improve performance while keeping portability, check Merging CPU Features for more information.
ignore: Do not record the CPU features used by the JVM in the image. This is mainly useful for troubleshooting if CPU feature handling is causing issues:
```
 $JAVA_HOME/bin/java -XX:CRaCCheckpointTo=cr \
    -XX:CPUFeatures=ignore \
    -jar target/spring-boot-0.0.1-SNAPSHOT.jar
```
Use this mode with care, as it weakens restore compatibility guarantees.

Please provide feedback to the CRaC project in such a case.

Explicit CPU feature value format

-XX:CPUFeatures can be set to an architecture-specific string encoding specific features. The format currently is two hexadecimal bitmasks separated by a comma, for example 0x1234567890abcdef,0xcdef.

Each bit roughly corresponds to one CPU capability flag (similar in spirit to /proc/cpuinfo flags), but exact bit-to-feature mapping is implementation-specific and may evolve.

-XX:CheckCPUFeatures=<value>

Sets the requirements for the CPU features check. Available values:

compatible: The image must be runnable on this CPU (default).
exact: The image must use the very same features as this CPU.
skip: Don’t check features at all. This option is allowed only when -XX:+UnlockExperimentalVMOptions is used.

In some cases, you can see the following error:

 $JAVA_HOME/bin/java -XX:CRaCRestoreFrom=cr

You have to specify -XX:CPUFeatures=0x21421801fcfbd7,0x3e6 together with -XX:CRaCCheckpointTo when making a checkpoint file; specified -XX:CRaCRestoreFrom file contains CPU features 0x4ff7fff9dfcfbf7,0x3e6; missing features of this CPU are 0x4de3de79c000020,0x0 = 3dnowpref, adx, avx512f, avx512dq, avx512cd, avx512bw, avx512vl, sha, avx512_vpopcntdq, avx512_vpclmulqdq, avx512_vaes, avx512_vnni, clflushopt, clwb, avx512_vbmi2, avx512_vbmi, rdpid, fsrm, gfni, avx512_bitalg, pku, ospke, avx512_ifma

If you are sure it will not crash you can override this check by -XX:+UnlockExperimentalVMOptions -XX:CheckCPUFeatures=skip .
<JVM exits here>

If you are confident that the missing CPU feature is not actually required by the JVM, you can force the JVM to run with -XX:+UnlockExperimentalVMOptions -XX:CheckCPUFeatures=skip. The JVM may still crash later with a SIGILL fault (on UNIX) or another fatal error (on Windows) because of the missing CPU feature or features. You have been warned!

 $JAVA_HOME/bin/java -XX:CRaCRestoreFrom=cr \
    -XX:+UnlockExperimentalVMOptions \
    -XX:+IgnoreCPUFeatures

-XX:+ShowCPUFeatures

You may use -XX:+ShowCPUFeatures to easily detect parameters for the -XX:CPUFeatures option on the computer intended to run -XX:CRaCRestoreFrom=PATH. Using --version as in this example is not mandatory, but otherwise the CPU feature output may scroll away.

 $JAVA_HOME/bin/java -XX:+ShowCPUFeatures --version
This machine's CPU features are: -XX:CPUFeatures=0x4ff7fff9dfcfbf7,0x3e6
CPU features being used are: -XX:CPUFeatures=0x4ff7fff9dfcfbf7,0x3e6
openjdk 22-internal 2024-03-19
OpenJDK Runtime Environment (fastdebug build 22-internal-adhoc.azul.crac-git)
OpenJDK 64-Bit Server VM (fastdebug build 22-internal-adhoc.azul.crac-git, mixed mode)