Visit Azul.com Support

Modifying Zing Memory Configuration

Table of Contents
Need help?
Schedule a consultation with an Azul performance expert.
Contact Us

If you completed all the steps in Installing and Configuring Azul Zing System Tools, Zing Memory is configured with default settings. Default configuration sets aside 75% of the physical memory of your machine for use by Azul Zing Builds of OpenJDK (Zing) when you use the reserve-at-config policy and allows up to 90% physical memory use for Zing when you use the reserve-at-launch (elastic) policy.

Zing uses Azul Zing System Tools (ZST) ZMM for managing memory for the three memory types that Zing uses:

  • Reservable memory – Memory than can be reserved by a Zing instance for its Java Heap.

  • Contingency memory – Temporary memory to use when Java Heap memory is low – a shared system-wide pool of memory that any Zing can use when the original value of –Xmxunderestimates the actual Java heap use.

  • Pause Prevention memory – A pool of System Zing Memory used to provide memory to Zing when there is a sudden increase in allocation rates, The use of the additional memory ensures the application threads can continue to allocate objects while the Continuously Concurrent Compacting Collector (C4) completes its work to free no objects that are no longer reachable.

The memory for these pools, collectively called System Zing Memory is configured when the ZST’s system-config-zing-memory tool is run. Memory can be reserved at configuration when the tool is run by using the reserve-at-config policy. Alternatively, just an upper limit can be set and no System Zing Memory reserved when the tool is run by using the reserve-at-launch policy. In this case, the allocation of System Zing Memory is postponed until the launch of a Zing process

For both configuration policies, at the launch of Zing, Zing requests that memory be reserved for its exclusive use as the process’s Java Heap – Zing’s Reserved memory. The C4 collector manages Java objects in the Java Heap.

When free memory in the Java Heap is low, Zing can use memory in the Contingency and Pause Prevention memory. When memory available for object allocation is first low, Zing uses Pause Prevention memory to prevent pauses that would normally occur when a thread tries to allocate an object but no memory is available. If Zing requires additional memory on an ongoing basis then memory from Contingency memory is used. This means that your Java application is no longer limited to the maximum heap size specified on the Java command line when you started your Java application.

Using the System Method to Allocate Zing Memory

To select the System method for allocating Zing Fund memory:

  1. From the system-config-zing-memory tool, specify System method, type: S

     
    ** Enter (s)ystem or (j)ava heap size [default 's']: s

    The system responds.

     
    [ Current task: Sizing Zing Memory / Choosing system sizing method ]

Do you want to allocate a percentage of your total host system memory to Zing, or would you prefer to specify an exact amount?

** Enter (p)ercentage or (e)xact [default 'p']:

  2. Select a memory allocation method:

    • Percentage – allocate a percentage of the total host system memory. To use this method, continue to Use Percentage of System Memory for Zing Memory.

    • Exact – allocate a specific amount of memory in MB. This is based on the sum of all of the Xmx values for all of the Zing instances that will be running concurrently. To use this method, continue to Use Exact Amount of System Memory for Zing Memory.

Use Percentage of System Memory for Zing Memory

To specify the amount of memory to allocate as a percentage of the total host system memory.

  1. In the system-config-zing-memory tool, specify the allocation method, type P

     
    ** Enter (p)ercentage or (e)xact [default 'p']: p

    The system responds with the next required configuration prompt.

    • Specify the percentage value to use, type a numeric value that represents the percentage. The default is 75% of Host System memory.

       
      ** Enter percentage of total memory to dedicate to Zing [default '75']:

      The system responds with the next configuration prompt.

  2. Continue to Specify Contingency Memory.

Use Exact Amount of System Memory for Zing Memory

To specify allocation amount as an exact value:

  1. From the system-config-zing-memory tool, specify the exact method, type: e

     
    ** Enter (p)ercentage or (e)xact [default 'p']: e

    The system responds.

     
    [ Current task:  Sizing Zing Memory / Choosing specific size ]

Please specify the amount of memory for Zing to use. (Note:  your system has a total of 15g.)

examples:  16384m, 16g

** Enter memory to dedicate to Zing:

  2. Specify the exact KB, MB, or GB value to allocate to Zing Memory. For example:

     
    ** Enter memory to dedicate to Zing: 12g

    The system responds with the next configuration prompt.

  3. Continue to Specify Contingency Memory.

Use Java Method to Allocate Zing Memory

To select the Java method for allocating Zing Memory.

  1. From the system-config-zing-memory tool, specify the exact method, type: e

     
    ** Enter (p)ercentage or (e)xact [default 'p']: e

    The system responds.

     
    [[ Current task: Sizing Zing Memory / Deriving size from Xmx ]

What is the sum of the Xmx values of all the Java processes you plan to run concurrently?

** Enter sum of Xmx:

  2. Respond with a size that is the maximum value for the sum of the Xmx values for all of the Zing instances that will run concurrently. For example (k=kB, m=MB, g=GB):

     
    ** Enter sum of Xmx: 2g

    The system responds with the next configuration prompt.

Specify Memory Reservation Policy

In pre-5.21.x ZST versions, you You can specify the policy that you want to use for memory reservation, type c or l the system-config-zing-memory tool. See the following example:

 
[ Current task: Choose memory reservation policy ]

You can reserve all of the requested System Zing Memory now using the

reserve-at-config policy or you can reserve the System Zing Memory for

each Zing VM process at process launch using the reserve-at-launch policy.

If you choose reserve-at-launch then the amount of memory reserved in the

previous step will be the upper limit available for all Zing VM\'s to use.

Which reservation policy do you want to use?

** Enter reserve-at-(c)onfig or reserve-at-(l)aunch [default 'c']:

Specify Contingency Memory

To specify the memory allocation for the Zing Contingency memory:

  1. From the system-config-zing-memory tool, type the desired percentage. For example: 75

     
    [ Current task: Sizing Zing Memory / Sizing Contingency memory pool ]

Part of Zing Memory is set aside in a common contingency memory pool for all Azul Java processes to share.  The contingency memory pool exists as an insurance policy to allow a JVM instance to temporarily avoid OOM behavior and grow beyond -Xmx.

What percentage of Zing Memory would you like to dedicate to the contingency memory pool?

** Enter percentage of Zing Memory to be used for contingency memory [default '5']:

Verify that the Memory Reservation is What You Expected

Verify that Zing Memory is restarted. View the output after the last step and locate the following lines. If you do not see them, see Starting or Stopping the Zing System Tools Services.

 
[ Current task:  Saving configuration and initializing Zing Memory module ]

zing-memory: INFO: Restarting...

zing-memory: INFO: Stopping...

zing-memory: INFO: stop successful

zing-memory: INFO: Starting...

Info: azulPmemPages: 54454.

Info: Zing Memory reserved using reserve-at-config policy.

INFO: az_pmem_reserve_pages (num2mPages 54454) succeeded

INFO: az_pmem_fund_transfer (to 7, from 0, bytes 114198315008) succeeded

INFO: az_pmem_fund_transfer (to 1, from 7, bytes 5708447744) succeeded

INFO: az_pmem_fund_transfer (to 3, from 7, bytes 5708447744) succeeded

INFO: az_pmem_fund_transfer (to 0, from 7, bytes 102781419520) succeeded

Info: You can now run Java processes up to -Xmx98020m or -Xmx95g.

Info: Azul pmem initialized successfully.

zing-memory: INFO: start successful

zing-memory: INFO: restart successful

Memory Configuration with 5.21.0 ZST and Later Versions

The system-config-zing-memory tool’s workflow is changed in the 5.21.0 ZST to support reserve-at-launch for all of the memory pools used by Zing, Reservable, Contingency, and Pause Prevention.

The workflow starts with a question that asks whether you want to configure memory as reserve-at-config (immediately as the tool is running) or reserve-at-launch (when the Zing process starts). Here is an example of the workflow:

The first section is a description of what the tool does – configure your use of System Zing Memory and create a file with that configuration as: /etc/zing/pmem.conf.0

 
Welcome to the Zing Memory configuration wizard by Azul Systems.

This wizard sets up the Zing Memory configuration file

/etc/zing/pmem.conf.0

and initializes System Zing Memory. Read the man pages for zing, zing-zst, and zing-pmem-conf for more information about this configuration file.

Use of System Zing Memory by Java processes running on the Zing VM requires you to configure memory in advance. This is analogous to reservation of memory for Linux hugepages, although System Zing Memory pages are not interchangeable with hugepages.

For most environments running Java applications on the Zing VM, Azul recommends setting up your system with an overall upper limit of 90% Zing Memory when using reserve-at-launch and a maximum of 75% when using reserve-at-config.

The second section starts the collection of the configuration information. The first step is determining which memory reservation policy you want to use, reserve-at-config or reserve-at-launch. There is no default value, you must select one or the other. Here is the explanation:

 
Current task: Memory reservation policy ]

You can reserve all of the requested System Zing Memory during this

configuration using the reserve-at-config policy or you can delay

reserving the System Zing Memory for each Zing VM process until process

launch using the reserve-at-launch policy. If you choose reserve-at-launch

then the amount of memory reserved in a subsequent step will be the overall

upper limit available for all Zing VMs to use.

The reserve-at-config policy will reserve the amount of memory you specify. The reserve-at-launch policy will prompt you for the UpperLimit for the amount of memory for each memory pool, Reservable, Contingency and Pause Prevention. You will also be prompted for an OverallUpperLimit.

The following table highlights the four major configuration types for 5.21.0 ZSTs and later versions:

Configuration Policy Specify Percentage of System Memory Specify Specific Amount

reserve-at-config

Must select percentage

Default

reserve-at-launch

Default

Must select specific

Reserve-at-config with percentage of host system memory:

 
Which reservation policy do you want to use?

** Enter reserve-at-(c)onfig or reserve-at-(l)aunch: c

** Enter (s)pecific amount or (p)ercentage of system memory [default 's']: p

** Enter Reservable memory as percent of system memory [default '68']:

** Enter amount of Contingency memory as % of system memory [default '4']:

** Enter amount of Pause Prevention memory as % of system memory [default '4']:

Reserve-at-config with specific amount of host system memory:

 
Which reservation policy do you want to use?

** Enter reserve-at-(c)onfig or reserve-at-(l)aunch: c

** Enter (s)pecific amount or (p)ercentage of system memory [default 's']:

** Enter amount of Reservable memory [default '<calc 68%>']:

** Enter amount of Contingency memory [default '<calc 4%>']:

** Enter amount of Pause Prevention memory [default '<calc 4%>']:

Reserve-at-launch percentage of host system memory:

 
Which reservation policy do you want to use?

** Enter reserve-at-(c)onfig or reserve-at-(l)aunch: l

** Enter (s)pecific amount or (p)ercentage of system memory [default 'p']:

** Enter upper limit for Overall Zing Memory use as % of system memory [default '90']:

** Enter upper limit for Reservable memory as % of system memory [default '85']:

** Enter upper limit for Contingency memory as % of system memory [default '50']:

** Enter upper limit for Pause Prevention memory as % of system memory [default '50']:

Reserve-at-launch specific amount of host system memory:

 
Which reservation policy do you want to use?

** Enter reserve-at-(c)onfig or reserve-at-(l)aunch: l

** Enter (s)pecific amount or (p)ercentage of system memory [default 'p']: s

** Enter upper limit for Overall Zing Memory use [default '<calc 90%>']:

** Enter upper limit for Reservable memory [default '<calc 85%>']:

** Enter upper limit for Contingency memory [default '<calc 50%>']:

** Enter upper limit for Pause Prevention memory [default '<calc 50%>']:

As with the pre-5.21.0 ZST system-config-zing-memory, the tool will output a description of the actions that it has taken on your behalf. Check the results to make certain that you have created the configuration you wanted.

Reservation Policies

When you run system-config-zing-memory on systems where you can use the reserve-at-launch policy, you have the choice to reserve all of the requested System Zing Memory immediately or just set an upper limit on the amount of Total System Memory to use as System Zing Memory. The following table compares these two policies that you can use to configure the System Zing Memory.

Reservation Policy Reserve System Zing Memory when system-config-zing-memory is run Easy to estimate free System Zing Memory available to Zing at Zing launch

reserve-at-config

Yes

Yes

reserve-at-launch

No

No

When run on a system (or partition) configured by using the reserve-at-config policy, the amount of memory available can be easily estimated with the command zing-ps -s.

When you run on a system (or partition) by using the reserve-at-launch policy, you must use both the zing-ps -s command and the information in /proc/meminfo to determine the amount of free Total System Memory available. Once calculated, use the output of zing-ps -s to determine the value for the upper limit configured for the maximum amount of Total System Memory available to use as System Zing Memory. The amount of Total System Memory in use will be influenced by the size of the page cache and the “Buffers:” and "Cached:” values, visible in /proc/meminfo, as shown below:

  • Buffers: 117772 kB

  • Cached: 2442412 kB

  • SwapCached: 0 kB

  • Active: 749576 kB

  • Inactive: 1830156 kB

If free memory is not available to launch a Zing when using reserve-at-launch, you can reclaim some of the memory used by the page cache by running the following:

 
free

Should print something similar to:

 
total used free shared buffers cached

Mem: 99056712 1556188 97500524 0 214464 341124

-/+ buffers/cache: 1000600 98056112

Swap: 4194296 0 4194296

Then run:

 
sync

/bin/echo 3 > /proc/sys/vm/drop_caches

Check free again to see whether you’ve reclaimed sufficient space.