Enabling Transparent Huge Pages for Azul Zing
|
Note
|
This setting is only applicable to Azul Zing Builds of OpenJDK (Zing) running without Azul Zing System Tools (ZST). Zing in ZST mode always uses its efficient large pages implementation. |
When you run applications on Azul Zing on x86_64 systems, we recommended configuring Transparent Huge Pages (THP) for Shared Memory. Note that as of Azul Zing 25.02.0.0, you do not need to enable THP for Shared Memory on ARM64 systems. If you are using ARM64 system architecture, we recommend using -XX:+UseTaggedAddressForJavaHeap in your command line options instead of configuring THP for Shared Memory.
We recommend configuring THP for Shared Memory on x86_64 systems for the following reasons:
-
Processes that use Linux Shared Memory, including Azul Zing, require a non-default THP configuration to be able to deliver additional performance benefits.
-
By default, OpenJDK uses THP without Shared Memory. While this offers some performance benefits, it suffers from indeterministic pauses due to defragmentation.
-
You can get a 5 to 15% performance boost for Azul Zing when THP is configured for Shared Memory.
Background Information
Transparent Huge Pages (THP) is a memory management feature of the Linux operating system, focusing on improving the system performance. It takes advantage of a concept introduced back in 2007 called Linux HugePages, which also sought to improve application performance. However, for an application to take advantage of the benefits provided by Linux HugePages, applications would need to explicitly include instructions to do so in their codebase and administrators would have to statically configure the number of required HugePages on the system. The introduction of Transparent Huge Pages sought to enable HugePages automatically thus making this transparent to the application which would not need to concern itself with these implementation details.
Java Virtual Machines (JVMs) can run just fine with THP either ENABLED or DISABLED, thus it is not a mandatory setting or prerequisite for running JVMs including Zing. However, HugePages result in lesser misses on the Translation Lookaside Buffer (TLB) cache in the CPU. TLB misses are generally handled by the hardware (with a hardware page table walker), but they incur a cost each time. An entry in the TLB for a 2MB (huge) page covers 512x memory compared to an entry in TLB for a 4KB (small) page. This range per entry represents the amount of “hotly accessed heap memory” for which the CPU can keep virtual-to-physical memory mappings cached in TLB entries.
Default Non-ZST Mode
When you are running Zing in the default mode, Zing employs multi-mapping for Java heap to implement the pauseless C4 garbage collection algorithm. The multi-mapping is done using memfd which uses Shared Memory related code in Linux. Hence, for Zing to be able to use HugePages for Java heap memory, THP for Shared Memory needs to be enabled. With THP for Shared Memory DISABLED the Java heap ends up using 4KB mappings. With THP ENABLED (including for Shared Memory) the Java heap can use 2MB mappings. The difference between those two are reflected in the pressure on the CPU TLB cache, and in the number of Data Translation Lookaside Buffer (DTLB) or Shared Translation Lookaside Buffer (STLB) misses incurred during operation. In practice, we have seen that enabling THP gives anywhere between 0% to 15% improvement in median latency and eventual CPU usage. The improvement depends on the workload. E.g., most micro benchmarks see no improvement at all, since their “hotly accessed heap memory” amount tends to be exceedingly small. While most “real” workloads see at least some improvement, with ~10% being commonly seen in our customer base.
Comparing THP With and Without Shared Memory
Transparent Huge Pages (THP) for Anonymous memory is ENABLED by default; whereas THP for Shared Memory is not. Additionally, THP support for Shared Memory is available on this Operating Systems.
Since Zing implements it’s C4 GC algorithm using memfd, Anonymous THP is not helpful, hence ENABLING Shared Memory for THP is necessary. OpenJDK may already be benefiting from THP for Anonymous Memory. However, the benefit depends on how well the application uses 2MB Virtual Pages.
Note that enabling THP for Shared Memory does not have any impact on applications that do not employ Shared Memory including other JVMs. In general, though, when enabling THP (for Anonymous or Shared Memory) it is important to make sure that the settings do not cause a lot of compaction stalls for the application. Azul has investigated the behavior of THP and has come up with the most appropriate settings that avoid these pitfalls, described below.
Prerequisites for Enabling Shared Memory THP
To enable shared memory THP, you need a Linux distribution as listed in the supported Operating Systems.
You can verify that your system can be configured for the type of THP supported by Zing, with the following command:
cat /sys/kernel/mm/transparent_hugepage/shmem_enabled
The returned line indicates the current enabled value, and must look like the following:
always within_size advise never deny force
Enabling Transparent Huge Pages
Once the above settings are in place, Zing in the non-ZST mode automatically uses transparent huge pages.
Anonymous memory for THP can be enabled by setting either madvise or always in /sys/kernel/mm/transparent_hugepage/enabled. The recommendation is to use the madvise setting so that only applications or JVMs that can take advantage of Anonymous Memory for THP use them. This reduces memory wastage and defragmentation.
Shared memory for THP can be ENABLED by setting either advise or always in /sys/kernel/mm/transparent_hugepage/shmem_enabled. The recommendation is to use the advise setting so that only applications or JVMs that can take advantage of Shared Memory for THP use them. This again reduces memory wastage and defragmentation.
|
Note
|
For Shared Memory THP to be ENABLED, Anonymous Memory THP also needs to be ENABLED. Moreover, Zing uses Anonymous THP for code cache and some other JVM internal structures that do not exist in the Java heap. |
Recommended Settings
When you are running Zing in the default mode, we recommend configuring THP for Shared Memory with the following commands:
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo advise | sudo tee /sys/kernel/mm/transparent_hugepage/shmem_enabled
echo defer | sudo tee /sys/kernel/mm/transparent_hugepage/defrag
echo 1 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/defrag
To persist these settings over a reboot, refer to Persisting the Configuration.
Non-ZST Mode with Tagged Address for Java Heap
When you are running Zing in non-ZST mode with UseTaggedAddressForJavaHeap enabled on ARM64 systems, Zing does not need shared memory THP to be configured. Instead, Zing makes use of regular anonymous memory THP. On most modern Linux distributions, anonymous THP is enabled by default. Therefore, Zing is able to make use of huge pages without the need for any additional kernel tuning.
Prerequisites for Enabling Tagged Address for Java Heap
The UseTaggedAddressForJavaHeap feature is only compatible with systems using ARMv8 system architecture or higher and Linux kernel versions 5.4 or higher. It is not compatible with x86_64 system architecture.
Recommended Settings
When you are running Zing in non-ZST mode with UseTaggedAddressForJavaHeap enabled, we recommend configuring THP with the following settings to reduce defragmentation stalls:
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo defer | sudo tee /sys/kernel/mm/transparent_hugepage/defrag
echo 1 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/defrag
Without using the recommended settings listed here, you still must make sure that /sys/kernel/mm/transparent_hugepage/enabled is set to either "always" or "madvise" and not set to “never”.
To persist these settings over a reboot, refer to Persisting the Configuration.
Controlling Defragmentation
Setting defer in sys/kernel/mm/transparent_hugepage/defrag results in defragmentation being handed over to background kernel daemons rather than doing this synchronously when the application asks for it. This means that if the application does not acquire a HugePage (when the system is fragmented) it simply continues with a small page rather than stall and wait for defragmentation to complete. Compaction gets initiated by kcompactd and HugePages are made available by khugepaged to the process.
Setting 1 in /sys/kernel/mm/transparent_hugepage/khugepaged/defrag enables defragmentation by the khugepaged daemon. Defragmentation (or compaction) helps acquiring more HugePages, especially after several days of system uptime.
More details about these configuration parameters can be found in the Linux kernel documentation for transparent huge pages.
THP in Containers and Kubernetes
Docker container images itself don’t need any configuration settings related to THP, as they use the configuration of the node/host, and for Zing, there are no additional JVM parameters required either. Transparent Huge Pages are, well, transparent, in the sense that the user process doesn’t know if the memory is using 2MB or 4KB mappings, and that this can change back and forth over time at the kernel’s discretion. With THP the most the JVM can do is hint that a mapping would prefer HugePages, using e.g., madvise, Zing does madvise on Java heap memory, as well as some off heap memory, by default if it detects that THP is enabled on the system.
Bottom line: THP control is a kernel-only, k8s node host-wide setting. Individual containers, pods, or images don’t need any (and have no way to set) per-container THP controls. The echo commands in the section above don’t work from non-privileged containers, as they can’t output to the /sys/kernel/… paths (even with sudo).
In the case where you are not able to SSH into the host or change the host’s configuration using start-up scripts, boot scripts, etc. there is an alternative, which is to use a k8s DaemonSet. A DaemonSet is guaranteed to run a pod on all nodes (that the set applies to, which is often just all the nodes in the cluster) before any application pods are started on those same nodes. As such, the pod that the DaemonSet runs would be considered a privileged pod that executes the needed kernel setting config commands. This is the simplest way to reliably make the change applicable everywhere in a cluster without having to touch anything in the host configuration and ensure that this setting "sticks."
You can ask Azul Support, [email protected], for more details and to help you with such a configuration.
Verifying Transparent Huge Pages
Once the settings are in place, Zing uses THP, and this can be verified in different ways.
Checking in Garbage Collector Logs
You can validate the THP usage in the Garbage Collector (GC) log header file. You can grep or search for "transparent huge pages", and should see the following:
[GCHH : Use transparent huge pages for java heap : true ]
Using meminfo
If the process is running and can successfully acquire HugePages, it should be reflected in the output of the following command:
$ grep HugePages: /proc/meminfo
AnonHugePages: 28672 kB
ShmemHugePages: 1017856 kB
FileHugePages: 0 kB
This lists Zing’s usage of THP for the complete Java heap defined by -Xmx under the ShmemHugePages metric.
Disabling Transparent Huge Pages
If you ever need to disable THP on Zing under this system configuration, add -XX:-UseTransparentHugePages to the command line of the Java process.
Persisting the Configuration
To persist the THP settings over a reboot, follow these steps according to your operating system.
You can persist the Transparent Huge Page configuration shown above with the following procedure, depending on your type of operating system.
On Ubuntu
-
Check if the setting is needed as it might already be enabled, and make a copy of this current setting:
/opt/zing/zing-jdk17/bin/java -Xlog:gc -version | grep huge > zingthp_old_ubuntu.conf -
Compare the settings with the recommendation shown above. If some of the settings already match and some not, your system is most likely tuned for some application. Refer to the documentation of that tuning to decide how to proceed and merge both tunings.
-
To permanently enable recommended THP settings for Zing without ZST, run the following:
$ sudo apt install sysfsutils $ sudo tee /etc/sysfs.d/zingthp.conf <<EOF kernel/mm/transparent_hugepage/enabled=madvise kernel/mm/transparent_hugepage/shmem_enabled=advise kernel/mm/transparent_hugepage/defrag=defer kernel/mm/transparent_hugepage/khugepaged/defrag=1 EOF $ sudo systemctl restart sysfsutils -
Verify the change:
/opt/zing/zing-jdk17/bin/java -Xlog:gc -version | grep hugeA sample output showing Zing using THP should look as follows:
[GCHH : Use transparent huge pages for java heap : true ]
On RHEL, CentOS, Oracle Linux, and Amazon Linux 2
To persist the THP configuration on CentOS Stream / RHEL 8 or newer, Oracle Linux 8 and 9 and UEK 7.9 or Amazon Linux 2:
-
Check if the setting is needed as it might already be enabled, and make a copy of this current setting:
/opt/zing/zing-jdk17/bin/java -Xlog:gc -version | grep huge > zingthp_old_rhel.conf -
Compare the settings with the recommendation shown above. If some of the settings already match and some not, your system is most likely tuned for some application. Refer to the documentation of that tuning to decide how to proceed and merge both tunings.
-
List the current active tuned profiles:
sudo tuned-adm activeIf this fails due to
tunednot being installed, install it with:sudo yum install tuned sudo systemctl enable tuned sudo systemctl start tunedAnd follow these steps to create the configuration:
$ sudo mkdir /usr/lib/tuned/zingthp $ sudo tee /usr/lib/tuned/zingthp/tuned.conf <<EOF [main] summary=THP configuration for Azul Zing # remove # from the "include" line and add the profile shown with the following command: # sudo tuned-adm active #include= [sysfs] /sys/kernel/mm/transparent_hugepage/enabled=madvise /sys/kernel/mm/transparent_hugepage/shmem_enabled=advise /sys/kernel/mm/transparent_hugepage/defrag=defer /sys/kernel/mm/transparent_hugepage/khugepaged/defrag=1 EOF -
Enter the existing profile name listed in the previous step into the configuration file in the line
include=. -
Enable the Zing THP configuration permanently:
sudo tuned-adm profile zingthp -
Verify the change:
/opt/zing/zing-jdk17/bin/java -Xlog:gc -version | grep hugeA sample output showing Zing using THP should look as follows:
[GCHH : Use transparent huge pages for java heap : true ]
On Amazon Linux 2023
To persist the THP configuration shown above on Amazon Linux 2023:
-
Check if the setting is needed as it might already be enabled and make a copy of this current setting:
/opt/zing/zing-jdk17/bin/java -Xlog:gc -version | grep huge > zingthp_old.conf -
Compare the settings with the recommendation shown above. If some settings already match and some not, your system is most likely tuned for some application. Refer to the documentation of that tuning to decide how to proceed and merge both tunings.
-
To permanently enable recommended THP settings for Zing, run the following:
$ sudo tee /etc/systemd/system/zingthp.service <<EOF [Unit] Description=SHMEM-THP configuration for Azul Zing Requires=local-fs.target Before=network.target [Service] Type=oneshot RemainAfterExit=true ExecStart=/bin/sh -c 'echo advise > /sys/kernel/mm/transparent_hugepage/shmem_enabled; \ echo defer > /sys/kernel/mm/transparent_hugepage/defrag' ExecStop=/bin/sh -c 'echo never > /sys/kernel/mm/transparent_hugepage/shmem_enabled; \ echo madvise > /sys/kernel/mm/transparent_hugepage/defrag' [Install] WantedBy=multi-user.target EOF $ sudo systemctl enable zingthp $ sudo systemctl start zingthp -
Verify the change to check if Zing uses THP for the Java heap:
$ /opt/zing/zing-jdk17/bin/java -Xlog:gc -version | grep huge [GCHH : Use transparent huge pages for java heap : true ]
|
Note
|
By default, Amazon Linux uses madvise for the defrag setting to enforce THP allocation which can lead to a pausing Java process. The preferred setting for Zing is defer which lets the khugepaged process allocate THP in the background without holding up the Java process. Alternatively, the value defer+madvise can be used for defrag, by changing the above configuration file if your system requires the madvise part.
|