General Issues Known in Azul Zing Builds of OpenJDK

Large Amounts of Virtual Memory Shown by top and ps for Zing Java Processes

Linux tools like top, ps, etc. show large amounts of virtual memory address space for all Zing Java processes in the range of several TBytes. This is an expected and normal behavior as Zing utilizes a large address space for its GC algorithm. It will not take away memory resources from other processes as the memory itself is not used, just the large address space is needed.

Unexpected Resident Set Size Memory Metric Shown by top and ps for Zing Java Processes

For Zing without the Azul Zing System Tools (ZST) component (the default mode of Zing), the Linux top, ps, and other tools overestimate the memory use of Zing Java processes. As a result, resident set sizes (RSS, RES) of more than 3 times the -Xmx value display. The reason for this is Zing’s usage of virtual memory multi-maps, i.e., multiple references to the same physical memory locations. To get precise metrics for the actual memory utilization, you can run the Linux smem -P java command which shows the memory usage in the PSS (proportional set size) column. However, you should only use this tool interactively for diagnostic purpose and not in a scheduled way on production systems to avoid a performance impact. There is a general latency performance impact on Linux systems when accessing the /proc/PID/smaps metric which is read by the smem tool and similar tools like atop and process-exporter, depending on their configuration. The better alternative to get an overview of the complete system memory condition is the Linux free command or /proc/meminfo as those show correct metrics without impacting performance.

For Zing with the installed ZST component, the memory metric resident set size (RSS, RES) does not include the memory used for the Java heap and thereby shows an unexpectedly small value of only a few hundred MBytes even when you set multiple GBytes of -Xmx. The reason for this is the dedicated memory management by the ZST. For performance reasons, ZST handles the memory for Java heaps independently of Linux default memory management. Only the total amount of memory used for Java heaps is visible by standard Linux tools like, for example, shown with the free command. To list the heap memory usage for Zing Java processes when ZST is installed, use the zing-ps -s command. With that, the heap memory, managed by the ZST, is shown in the Xmx column, the small amount of Linux RSS - in the LRSS column, and the memory used by Zing internally and managed by the ZST - in the ZRSS column.

Also, on Zing with the installed ZST, a similar effect is visible regarding the Linux metric Mlocked in /proc/meminfo. While the ZST-managed memory is protected from being swapped out, similar to a Linux mlock, it is not visible as Mlocked in /proc/meminfo.

Slow Process Restart for Large Heap Sizes on Small Linux Pages

In non-ZST mode on system with Linux standard configuration, i.e. disabled shared memory transparent huge pages, restarting the java process can take up to 10 seconds per 100 GB heap size. Solution: Enable shared memory transparent huge pages as described in Enable Huge Pages or install ZST. In addition, on Ubuntu, Amazon Linux or similar systems with Linux kernel 4.19.7 and newer upgrade to Azul Platform Prime 21.04.0.0 or newer.

Loop predicate and loop limit check code problem