Enhanced Linux Memory Defragmentation

You are here:

Enhanced Linux Memory Defragmentation

This section addresses the situation when Zing fails with one of the following error messages:

Memory management: unable to fund java heap account
ERROR: perror: az_pmem_reserve_pages (num2mPages ...

As these errors can occur when not enough free memory is available, i.e. other large processes are already running, first check with the command zing-ps -s and top if other Zing or other large processes are already running.

If enough free memory is available as indicated by the Linux free command, then the cause of these error messages is often memory fragmentation. Zing needs large Linux pages of at least 2 MB page size and those might be too fragmented.

If rebooting the system is not an acceptable solution in this case, upgrade to at least Zing Systems Tools ZST 5.21.7 and add the following line to the file /etc/zing/drop.conf:

RESERVE2MB=true

With this setting Zing's enhanced Linux memory defragmentation is enabled in Zing to be able to operate under situations of high fragmentation.

Note:

Do not enable the RESERVE2MB=true setting on RHEL/CentOS 6.4 and older versions or when running on Linux kernels older than 2.6.32-431. Due to a bug in those older Linux kernels the system can crash (see CentOS bug 6660).

Setting RESERVE2MB=true enables Linux memory defragmentation. This Linux function causes all processes on the host to stop for a few ms to seconds depending on the RAM size if defragmentation is triggered.

When using this setting with the ZST mode reserve-at-config, the memory initialization and potential triggering of the defragmentation happens only during the start of the ZST system service zing-memory. It is safe to enable this setting there if you do not restart the zing-memory system service during the operation of other business applications on your host.

Alternatively, you can run Zing without ZST (possible since ZVM 19.07.0.0) or update your operating system to a version with Linux kernel 4.4 or newer, where the RESERVE2MB setting is not needed anymore.

When enabling the RESERVE2MB setting in combination with the ZST mode reserve-at-launch, it can trigger the defragmentation potentially on each start of a ZVM Java process. Depending on the application use case, it is often acceptable, for example, if Java process starts occur during non-business hours only.

With reserve-at-launch and the RESERVE2MB setting, there exists an additional small risk of triggering the defragmentation even during runtime of a ZVM process when it consumed all the available contingency or pause prevention memory and is requesting more. To prevent triggering the defragmentation during runtime, set the pause prevention and contingency memory pools to a static value, for example, each to 3% of the system RAM in the file/etc/zing/pmem.conf and restart the zing-memory system service:

     ContingencyUpperLimit 3%
     ContingencyMinimumFreeMemory 3%
     PausePreventionUpperLimit 3%
     PausePreventionMinimumFreeMemory 3%