Visit Azul.com Support

Samples Memory Usage by Account

Need help?
Schedule a consultation with an Azul performance expert.
Contact Us

Examples 6 through 10 in the following subsections depict methods of using the zing-ps command to list information about Azul Zing Builds of OpenJDK (Zing) processes.

Example 1: Monitoring a Test Run.

If you run a while loop, logging it into a file every 10 seconds or so, you can then make graphs from the log data, using your favorite raw-number-to-nice-graphs conversion tool.

 
# while true; do zing-ps -s >> /tmp/zing-ps_log ; sleep 10; done

Example 2: Diagnosis of Zing Fault

In the rare occurrence of a Zing crash, it can be useful to collect the output of zing-ps –s and zing-ps --acctinternal (debug option) from the system after the crash occurs, but before a system reboot or a ZST restart. If Zing crashed, please contact your Azul representative who might direct you to collect the zing-ps output and upload to the Azul FTP site along with other data, logs and cores.

Example 3: Memory statistics with zero Zing processes running.

This example shows memory funding with no Zing processes running.

 
# zing-ps --acct Fund breakdown (total 8) NAME BALANCE MAXIMUM fund[0]: Committed 22745710592 22745710592 fund[1]: Overdraft 1262485504 1262485504 fund[2]: Unknown 0 0 fund[3]: PausePrevention 1262485504 1262485504 fund[4]: Unknown 0 0 fund[5]: Unknown 0 0 fund[6]: Unknown 0 0 fund[7]: Unknown 0 0 Found 0 processes

Example 4: Memory account details with 1 Zing process running.

 
$ zing-ps --acct2 Process level Maximum: 0 NAME BALANCE(now/min/max KB) ALLOCATED(now/min/max KB) MAXIMUM(KB) ACS_TO_CREDIT FND ODFND AC[0]:Default 32768/ 30720/ 32768 2048/ 0/ 4096 32768 0xffffffffffffff00 0 1 AC[1]:Unknown 0/ 0/ 0 0/ 0/ 0 0 0xffffffffffffff01 -1 -1 AC[2]:JavaHeap 11360/ 911360/ 1873920 2201600/ 1239040/ 2201600 8067072 0xffffffffffff0203 0 1 AC[3]:PausePrevention 0/ 0/ 0 0/ 0/ 0 4954112 0xffffffffffff0203 3 3

Field descriptions:

  • Values are KB bytes.

  • account [0]: The maximum is equal to the Partition. The allocated plus the balance are equal to maximum.

  • account [2]: Java Heap. If the balance is negative, the application is using contingency memory.

  • account [3]: Maximum value allocated for Pause Prevention.

The value for account[2] heap ALLOCATED (used) is higher than AC[2] javaHeap ALLOCATED now (used) because the zing-ps --acct (in these examples) was followed by the zing-ps --acct2 and the application continued to allocate objects into the Java heap.

Example 5: Memory funding details with 1 Zing process running.

This example shows memory funding breakdown for ZST and allocation to the Zing process PID 15479.

 
$ zing-ps --acct Fund breakdown (total 8) NAME BALANCEMAXIMUM fund[0]:Committed 819357286442553311232 fund[1]:Overdraft 23634903042363490304 fund[2]:Unknown 00 fund[3]:PausePrevention23634903042363490304 fund[4]:Unknown 00 fund[5]:Unknown 00 fund[6]:Unknown 00 fund[7]:Unknown 00 Found 1 processes. USER PID PROCESS root 15479 /opt/zing/zing-jdk1.7.0_23-4.2.0.0.beta-14-x86_64/bin/java NAMEBALANCEALLOCATEDMAXIMUMFNDODFND account[0]:default341835776209715234393292801 account[1]:emergency_gc000-1-1 account[2]:heap3122659328027892121603637929574401 account[3]:pause_prevention0027487790694433 account[4]:unknown000-1-1 account[5]:unknown000-1-1 account[6]:unknown000-1-1 account[7]:unknown000-1-1

Explanation of zing-ps Fund Breakdown

The Fund breakdown in Memory funding details with 1 Zing process running, shows the memory funding as set up for the ZST. The output from the command then also shows that the zing-ps acct command has found one Zing Java process PID 15749 and gives the memory breakdown for that Java process.

The table below takes a look at the example Fund Breakdown in more detail. The figures have been converted from bytes to megabytes (because it is easier to think about megabytes than large numbers of bytes).

NAME BALANCE MAXIMUM

fund[0]:

Committed

7814

40582

The Committed Maximum is the maximum Reservable Heap

The Committed Balance is the maximum Available Heap size for the JVM -Xmx command option; and if other JVMs have already been started this will be less than the maximum Reservable value.

Java Process ID 15479 has been started and set a –Xmx of 32768MB reducing the Available heap from 40582MB to 7814MB

fund[1]:

Overdraft

2254

2254

The Overdraft is the Contingency Memory (using the ZST defaults, this is 5% of the Zing Memory)

fund[3]:

PausePrevention

2254

2254

The PausePrevention is the Pause prevention memory (using the ZST defaults, this is 5% of the Zing Memory)

The sum of the Committed Maximum and the Overdraft and Pause prevention maximums is equal to the size of the Zing Memory (using the ZST defaults this is 75% of the available Operating System memory, leaving 25% for the Linux Memory). In this case 40852 + 2254 + 2254 = 45090MB from which we can infer that if the ZST was configured with a 75/25 allocation then the available Operating System memory at the time the ZST was configured was 60120MB (58.7GB). This can be useful to know; for example, if we made another assumption that this was a 64GB RAM machine, then 5416MB has been allocated away somewhere else before the Operating System was loaded – perhaps some DMA buffers for 10GBe NIC interfaces or some such.

Example Memory Fund Allocation for a Process

Carrying on with Memory funding details with 1 Zing process running, using Process 15479 for illustration.

The table below takes a look at the example Fund allocation in more detail. The figures have been converted from bytes to megabytes (because it is easier to think about megabytes than large numbers of bytes).

PID: 15479 /opt/zing/zing-jdk1.7.0_23-4.2.0.0.beta-14-x86_64/bin/java

NAME BALANCE ALLOCATED MAXIMUM FND ODFND

Account 0

default

326

2

328

0

1

The Account 0 value is in bytes, and it amounts to 328MB. Its max is equal to its funding (that is, the ALLOCATED plus the BALANCE are equal to max).

Account 0 is used for some ZST-based memory stuff that is not purely in the heap (such as kernel structures for holding the kernel page tables and such).

The JVM assigns 1% of -Xmx to Account 0, and the rest to Account 2 (both draw from Committed memory).

Account 1

emergency_gc

0

0

0

-1

-1

Account 2

heap

29780

2660

34694

0

1

Account 2 is the Java Heap. If its BALANCE shows as negative, your application is using contingency memory.

The Account 2 funding (BALANCE + ALLOCATED) together with Account 0 funding adds up exactly to the -Xmx value. In this case -Xmx ⇒ 29780 + 2660 + 328 = 32768MB

The option flag -XX:JavaMemMax can be used to set the maximum for Account 2 (heap).

NOTE: When -XX:JavaMemMax=<val> equals -Xmx<val>, then the Zing instance is not allowed to use any Contingency memory.

Account 3

pause_prevention

0

0

262144

3

3

The Account 3 MAXIMUM value for the pause_prevention memory computes to 256GB (exactly). Having a large MAXIMUM value doesn’t mean you have funding for that much. It just means you can draw up to that much total allocation (including dipping into overdraft accounts) if the ZST has resources to give you in the fund you are drawing from.

In the case of pause prevention, a max of 256GB just means that this process can draw down the entire pause prevention memory if it wants to (it has zero funding for account 3, so all of that draw shows up in negative balances equal to the allocation level).

The MAXIMUM values for each account are not funding numbers. They are just absolute limits on how much can be allocated in that account. This matters a lot when overdraft accounts are involved, as the MAXIMUM caps the total amount of allocation, even after the balance goes negative.

The committed funding is always the sum of BALANCE + ALLOCATED, this remains true including times when Zing dips into an overdraft account, such as contingency (the BALANCE will go negative at that point, and even though ALLOCATED is greater than committed funding, the sum is still equal to committed).

Zing uses Zing Memory only for the -Xmx (Java heap). So you need to leave enough Linux memory to accommodate not only the other Linux stuff that’s there, but also Zing’s C-heap.