Enabling Transparent Huge Pages for Azul Zing

Background Information

Transparent Huge Pages (THP) is a memory management feature of the Linux operating system, focusing on improving the system performance. It takes advantage of a concept introduced back in 2007 called Linux HugePages, which also sought to improve application performance. However, for an application to take advantage of the benefits provided by Linux HugePages, applications would need to explicitly include instructions to do so in their codebase and administrators would have to statically configure the number of required HugePages on the system. The introduction of Transparent Huge Pages sought to enable HugePages automatically thus making this transparent to the application which would not need to concern itself with these implementation details.

Java Virtual Machines (JVMs) can run just fine with THP either ENABLED or DISABLED, thus it is not a mandatory setting or prerequisite for running JVMs including Zing. However, HugePages result in lesser misses on the Translation Lookaside Buffer (TLB) cache in the CPU. TLB misses are generally handled by the hardware (with a hardware page table walker), but they incur a cost each time. An entry in the TLB for a 2MB (huge) page covers 512x memory compared to an entry in TLB for a 4KB (small) page. This range per entry represents the amount of “hotly accessed heap memory” for which the CPU can keep virtual-to-physical memory mappings cached in TLB entries. Even when there is a miss in the TLB fetching a 2MB mapping from the Linux hierarchical page table is cheaper than fetching a 4KB mapping.

Zing employs multi-mapping for Java heap to implement the pauseless C4 garbage collection algorithm. The multi-mapping is done using memfd which uses Shared Memory related code in Linux. Hence, for Zing to be able to use HugePages for Java heap memory, THP for Shared Memory needs to be enabled. With THP for Shared Memory DISABLED the Java heap ends up using 4KB mappings. With THP ENABLED (including for Shared Memory) the Java heap can use 2MB mappings. The difference between those two is reflected in the pressure on the CPU TLB cache, and in the number of Data Translation Lookaside Buffer (DTLB) or Shared Translation Lookaside Buffer (STLB) misses incurred during operation. In practice, we have seen that enabling THP gives anywhere between 0% to 15% improvement in median latency and eventual CPU usage. The improvement depends on the workload. E.g., most micro benchmarks see no improvement at all, since their “hotly accessed heap memory” amount tends to be exceedingly small. While most “real” workloads see at least some improvement, with ~10% being commonly seen in our customer base.

Comparing THP With and Without Shared Memory

Transparent Huge Pages (THP) for Anonymous memory is ENABLED by default; whereas THP for Shared Memory is not. Additionally, THP support for Shared Memory is available on this Operating Systems.

Since Zing implements it’s C4 GC algorithm using memfd, Anonymous THP is not helpful, hence ENABLING Shared Memory for THP is necessary. OpenJDK may already be benefiting from THP for Anonymous Memory. However, the benefit depends on how well the application uses 2MB Virtual Pages.

Note that enabling THP for Shared Memory does not have any impact on applications that do not employ Shared Memory including other JVMs. In general, though, when enabling THP (for Anonymous or Shared Memory) it is important to make sure that the settings do not cause a lot of compaction stalls for the application. Azul has investigated the behavior of THP and has come up with the most appropriate settings that avoid these pitfalls, described below.

Prerequisites for Enabling Shared Memory THP

You need a Linux distribution as listed in the supported Operating Systems.

You can verify that your system can be configured for the type of THP supported by Zing, with the following command:

 
cat /sys/kernel/mm/transparent_hugepage/shmem_enabled

The returned line indicates the current enabled value, and must look like the following:

 
always within_size advise never deny force

Recommended Settings

Configure THP for Shared Memory with the following commands, as explained further:

 
echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo advise | sudo tee /sys/kernel/mm/transparent_hugepage/shmem_enabled
echo defer | sudo tee /sys/kernel/mm/transparent_hugepage/defrag
echo 1 | sudo tee /sys/kernel/mm/transparent_hugepage/khugepaged/defrag

To persist these settings over a reboot, refer to Persisting the Configuration.

Verifying Transparent Huge Pages

Once the settings are in place, Zing uses THP, and this can be verified in different ways.

 
[GCHH : Use transparent huge pages for java heap : true ]

 
$ grep HugePages: /proc/meminfo
AnonHugePages:     28672 kB
ShmemHugePages:  1017856 kB
FileHugePages:         0 kB

Disabling Transparent Huge Pages

If you ever need to disable THP on Zing under this system configuration, add -XX:-UseTransparentHugePages to the command line of the Java process.

Persisting the Configuration

To persist the THP settings over a reboot, follow these steps according to your operating system.

You can persist the Transparent Huge Page configuration shown above with the following procedure, depending on your type of operating system.