Back to Zing Documentation Home

Issues Known in Previous Releases of Zing Virtual Machine

This section summarizes issues known in Zing.

Large Amounts of Virtual Memory Shown by top and ps for Zing Java Processes

Linux tools like top, ps, etc. show large amounts of virtual memory address space for all Zing Java processes in the range of several TBytes. This is an expected and normal behavior as Zing utilizes a large address space for its GC algorithm. It will not take away memory resources from other processes as the memory itself is not used, just the large address space is needed.

Unexpected Resident Set Size Memory Metric Shown by top and ps for Zing Java Processes

For the Zing Virtual Machine without the Zing System Tools (ZST) component (the default mode of Zing), the Linux top, ps, and other tools overestimate the memory use of Zing Java processes. As a result, resident set sizes (RSS, RES) of more than 3 times the -Xmx value display. The reason for this is Zing's usage of virtual memory multi-maps, i.e., multiple references to the same physical memory locations. To get precise metrics for the actual memory utilization, run the Linux smem -P java command which shows the memory usage in the PSS (proportional set size) column. The Linux free command shows correct summary metrics.

For Zing with the installed Zing System Tools component, the memory metric resident set size (RSS, RES) does not include the memory used for the Java heap and thereby shows an unexpectedly small value of only a few hundred MBytes even when you set multiple GBytes of -Xmx. The reason for this is the dedicated memory management by the ZST. For performance reasons, ZST handles the memory for Java heaps independently from Linux default memory management. Only the total amount of memory used for Java heaps is visible by standard Linux tools like, for example, shown with the free command. To list the heap memory usage for Zing java processes when ZST is installed, use the zing-ps -s command. With that, the heap memory, managed by the ZST, is shown in the Xmx column, the small amount of Linx RSS - in the LRSS column, and the memory used by Zing internally and managed by the ZST - in the ZRSS column.

Also on Zing with the installed ZST, a similar effect is visible regarding the Linux metric Mlocked in /proc/meminfo. While the ZST-managed memory is protected from being swapped out, similar to a Linux mlock, it is not visible as Mlocked in /proc/meminfo.

Version-Specific Known Issues

The following table lists issues known prior to the release of Zing Virtual Machine 20.09.1.0.

Release Known Description
20.07.0.0

Applications using munlockall() require
-XX:-UseThreadStateNativeWrapperProtocol on the command line to avoid crash or inconsistency if the rare situation occurs that the application gets swapped out after the munlockall() invocation.

20.05.1.0

ZVM crash is seen when there is a mismatch between ZST versions on a host and containers.

Workaround: Ensure ZST versions on a host and containers match. Also, note that since ZVM 19.07.0.0, Zing does not need the ZST component which was required in earlier versions of Zing. Starting with ZVM 19.07.0.0, install only a ZVM package inside your container to run Zing within a container.

20.04.0.0

Heap dumps and JVMTI object tagging are not supported with UseEpsilonGC.

19.12.102.0

The combination of the KeepCodeEntrantOnAsyncExceptions command-line option with the Tick Profiler can have issues.

19.10.0.0

Test failure due to code cache exhaustion on configurations with heaps of 4 GB and below are more likely to happen on ZVM 19.10.0.0 as the default CodeCache size progressively shrinks for such configurations.

Workaround: Increase the code cache size using the -XX:ReservedCodeCacheSize=<size> option (you can specify up to 1280 MB for <size>). Alternatively, switch off the FalconGenerateProfilerInfo flag, set to true by default, for improving observability and gathering more accurate and complete information for debugging.

19.06.0.0

When used in -XX:-UseZST mode, applications that allocate significant amounts of large (multi-MB) objects at high rates and on a sustained basis may observe higher than normal time to safepoint behaviors.

19.06.0.0

When Zing is used in the Non-ZST mode, the RSS/RES value reported by such Linux tools as top or ps are known to be inaccurate because multi-mapped regions are counted multiple times by those commands, even though they point to the same physical memory. The Linux free command shows the correct summary amount and the Linux smem -P java command shows the precise amount per process in column PSS.

19.04.0.0

Zing License Verification failed when /tmp is full/readonly.

Workaround: Clean up the /tmp directory.

19.04.0.0

Zing 11 JFR events refer to null bootstrap class loader as the class loader for anonymous classes, which is not accurate.

19.04.0.0

The Live Objects ZMC representation does not show anything for Zing 11 JFR recordings as the JFR Leak Profiler is not yet enabled in Zing 11 JFR and the OldObjectSample event is not generated.

Zing releases from 17.08.0.0 to 19.03.0.0

The -XX:+UseNUMA command-line option can cause reduced application performance when used on versions 17.08.0.0 - 19.03.0.0 of Zing VM.

If you have -XX:+UseNUMA active, the symptoms to look for are as follows:

  • very high system CPU time, mostly spent in kernel ioctl calls
  • several seconds long thread allocation delays visible over GCLogAnalyzer in the gc.log
  • high CPU load, even outside of GC cycles and without application activity

Workaround: Remove -XX:+UseNUMA from the command line or upgrade to ZVM 19.04.0.0.

18.12.0.0

In Java 11 testing, a rare ZVM 18.12 crash was observed when returning from the invocation of a MethodHandle routine. Analysis revealed the failure to be reproducible under rare circumstances in Java 8 with previous ZVM releases. Frequency of reproduction appears to be influenced by use of ReadyNow! with Java 11. It is recommended that users of ReadyNow! do not upgrade to Java 11 until this issue is resolved.

18.10.0.0

Handshake messages can be strictly ordered.

Workaround: Use alpn-boot-8.1.13.v20181017.jar.

18.09.0.0 When run on ZST 5.21.x and older, ZVM 18.09.0.0 cannot tick-profile native threads.
18.06.0.0 ZVM 18.06.0.0 can crash when -XX:+PreserveFramePointer is used.

18.03.0.0

18.03.1.0

In 18.03.0.0 and 18.03.1.0, the combination of ReadyNow!, compile stashing (enabled by default with ReadyNow!), and a profile which is fed from one run to another in a rolling manner can in some circumstances result in severe peak performance loss. If using ReadyNow! with a rolling profile on 18.03.0.0 or 18.03.1.0, it is strongly recommended to upgrade to 18.03.2.0, 18.04.0.0, or a later release which include fixes for this issue. If a peak performance problem is observed, we recommend discarding the profile collected with 18.03.0.0 or 18.03.1.0 and recollecting a fresh one on an unaffected version.

15.09.0.0

When using reserve-at-launch, you may see an exception such as "MemoryUsage ERROR: initialReserved 0 size -4738568192 used -4736471040" in Zing MXBean calls that use MemoryUsage objects for System Linux Memory. As of ZVM 16.01.0.0, the probability of seeing this error has been reduced. If the error is seen, it can safely be ignored.

15.09.0.0

WebSphere fails to launch when using -XX:+UseZingMXBeans. Workaround: Set the following by using server.xml or WAS admin console (note the empty value for the system property):

genericJvmArguments="-XX:+UseZingMXBeans -Djavax.management.builder.initial= "

all Zing releases    

Loop predicate and loop limit check code problem.

(1) The java spec says explicitly that operations on integers can overflow and no exception will be thrown. Indeed, unlike the C++ spec that says integer overflow is undefined, java says integer overflow is required to happen.

(2) Java coding guidelines always point out that using Integer.MAX_VALUE in comparison expressions is dangerous programming. Especially if it is used in a loop bound.

(2a) This loop will terminate because eventually i will be equal to Integer.MAX_VALUE: for (int i=0; i<Integer.MAX_VALUE; i+=1) { ... }

(2b) This loop will never terminate, because i will overflow: for (int i=0; i<Integer.MAX_VALUE; i+=2) { ... }

(3) Zing will sometimes terminate the loop in (2b).

Note: -XX:+DisableLoopOptimizations will often avoid the problem but is is only recommend as a workaround on a per-method basis.

-XX:CompileCommand='disableloopopts,classname::method'

15.09.0.0

JBoss7 throws an exception on startup with -XX:+UseZingMXBeans flag. Workaround: Add the following lines to the standalone.conf file:

JAVA_OPTS="$JAVA_OPTS -Djava.util.logging.manager=org.jboss.logmanager.LogManager"

JAVA_OPTS="$JAVA_OPTS -Xbootclasspath/p:../modules/org/jboss/logmanager/main/jboss-logmanager-1.2.0.GA.jar:../modules/org/jboss/logmanager/log4j/main/jboss-logmanager-log4j-1.0.0.GA.jar:../modules/org/apache/log4j/main/log4j-1.2.16.jar"

14.09.0.0

ZVM running in environment with multiple scheduling policies, RR, BATCH, and OTHER, encounters checkpoint sync timeout in thread running with BATCH policy.

 

See Also

 

 

Last modified: September 30, 2020


© Azul Systems, Inc. 2020 All rights reserved.

Privacy Policy | Legal | Terms of Use