Allows usage of the remote compiler when Cloud Native Compiler has established a connection. Requires OptHubHost.
Using Cloud Native Compiler
You configure an Azul Zing Build of OpenJDK (Zing) to request compilations from Cloud Native Compiler by specifying the IP address of the service along with other command-line options. If the Cloud Native Compiler cannot respond to the compilation requests in time, the Azul Zing JVM switches to local JIT compilation until the service recovers.
Fallback to Local JIT Compilation
When the Cloud Native Compiler component becomes unavailable or is unable to deliver compilations at acceptable speeds, the Zing JVM falls back to local compilation to protect application performance.
Fallback Reasons
When you connect a Zing JVM to a Cloud Native Compiler, the JVM attempts to fetch all JIT compilations from the service. If the Cloud Native Compiler cannot meet the JVM’s requests in time, the JVM automatically falls back to performing optimizations on the client. Factors that can trigger local fallback include:
-
The service does not have the corresponding Compiler Engine.
-
The service is down or the JVM cannot reach it.
-
The service is delivering compilations at an unacceptably slow rate for an extended period. This can occur in several scenarios:
-
Underprovisioned Optimizer Hub: The Hub does not have enough capacity to handle the number of incoming compilation requests. Too many clients connecting at the same time causes compilations to queue up, and application productivity can drop below what local JIT compilation would provide. The Zing JVM may switch to local fallback in this situation. See Sizing and Scaling your Optimizer Hub Installation for more info.
-
Slow scale-out during burst traffic: Even with autoscaling enabled, new resources take time to come online. During deployment waves or other sudden spikes in client connections, a large compilation backlog can build up before Optimizer Hub can caught up. Application productivity may temporarily drop during this period, and the Zing JVM may use local fallback until capacity recovers.
-
A new version of your application gets deployed with a new profile name: When a new application version is deployed under a different profile name, Optimizer Hub treats it as an unknown application and starts profiling from scratch. A newly started profile tends to have lower hit rates and slower optimizations for the first few instances involved.
-
When the JVM detects that Cloud Native Compiler is consistently delivering compilations slower than the client’s local JIT capabilities, it automatically switches to local fallback to maintain acceptable service levels.
Local Fallback Baseline and Configuration
Before deploying Cloud Native Compiler in production, you must establish a Local Fallback Baseline for your application. This baseline represents the acceptable performance characteristics when running with local JIT compilation and depends on the CPU resources you have allocated to your JVM instances.
Understanding Your Local Resource Classification
The viability of local fallback depends on the CPU resources available in your deployment for the JIT compiler threads:
-
No CPU/Minimal CPU: Some deployments have aggressively reduced CPU allocation after migrating to Cloud Native Compiler. In these cases, any use of local JIT can cause critical issues. If this describes your environment, you may need to disable performance-based fallback or ensure an adequate CPU is available for emergency fallback scenarios.
-
1 vCPU or limited compiler threads: This is the most common case for mature Cloud Native Compiler deployments. Local compilation will be slower than optimal but can serve as a disaster recovery mechanism.
-
Sufficient CPU for local JIT: Applications that recently migrated to Cloud Native Compiler or retained their original CPU allocation. These deployments can effectively use local fallback without service degradation.
Establishing Your Local Fallback Baseline
You need to verify two scenarios:
-
Cloud Native Compiler Success Criteria: Your application meets acceptable performance when Cloud Native Compiler is fully operational:
-
Response latencies meet your SLAs
-
CPU utilization is within expected ranges
-
No critical outages or problems when restarting fleets under normal operations
-
-
Local Fallback Baseline: Your application can survive when switching to local fallback:
-
Response latencies and CPU utilization are higher but still within acceptable limits
-
Time to full speed is longer, and the carrying capacity of each server is lower (requiring more instances)
-
No critical outages related to pods starting without fully operational Cloud Native Compiler
-
Configure your Local Fallback Baseline with -XX:CNCLocalFallbackOptLevelLimit=<3,2,1,0> and -XX:CIMaxCompilerThreads=<n> to tune local compilation behavior. For more information, check the Zing documentation > Falcon Compiler Options > -XX:FalconOptimizationLevel.
|
Note
|
You must have monitoring in place to measure response latencies, CPU utilization, and detect outages during fleet restarts. |
Performance-Based Local Fallback
The Zing JVM continuously monitors the rate at which Cloud Native Compiler delivers compiled code to the client. The JVM enters the local fallback when:
-
The number of compilations that Cloud Native Compiler delivers per second falls below a threshold based on your local compiler thread configuration and optimization level.
-
The remote compilation queue has reached a significant depth.
-
Both conditions persist for several seconds (default 5 seconds).
When in local fallback, the JVM performs both local and remote compilations simultaneously. This allows the JVM to:
-
Provide immediate compilation results from local resources.
-
Continue monitoring Cloud Native Compiler performance to detect when it recovers.
-
Use the first available compilation from either source.
The JVM exits local fallback and returns to Cloud Native Compiler-only mode when both of the following conditions are met:
-
The remote compilation queue clears below the threshold.
-
The rate of incoming code deliveries from Cloud Native Compiler consistently exceeds the local capability threshold.
Check the available Productivity flags in the list of Cloud Native Compiler JVM Options to configure Performance-Based Local Fallback.
Automatic Switch Back to Cloud Native Compiler
The following output in the JVM concomp log shows when the JVM enables and disables fallback to local JIT compilation:
[110,991s][info ][concomp] [LocalFallback] local compilation queue disabled
[111,018s][info ][concomp] [LocalFallback] local compilation queue enabled
When local fallback is active, the JVM continues to request and receive compilations from Cloud Native Compiler while also compiling locally. You may observe both local and remote compilation activity in the logs during this period.
Logging and SSL
To view compiler info and ensure that the JVM is correctly connecting to Optimizer Hub, use the -Xlog:concomp flag.
By default the Zing JDK connects to Optimizer Hub using SSL. If you did not enable SSL during Optimizer Hub deployment, you must use the -XX:-OptHubUseSSL flag to instruct Zing to connect without SSL.
If you attempt to connect to Optimizer Hub, running without SSL, and do not specify the -XX:-OptHubUseSSL flag, you get the following error (visible with the -Xlog:concomp flag):
E1011 13:16:23.198074100 29 ssl_transport_security.cc:1446] Handshake failed with fatal error SSL_ERROR_SSL: error:1408F10B:SSL routines:ssl3_get_record:wrong version number.
Cloud Native Compiler JVM Options
|
Note
|
The minimum JVM options to request compilations from Cloud Native Compiler are -XX:OptHubHost={value} and -XX:+CNCEnableRemoteCompiler. It’s also advised to specify -XX:ProfileName=<value> for improved Cloud Native Compiler caching.
|
| Command Line Option | Description | Default | ||
|---|---|---|---|---|
-XX:OptHubHost=<host>:<port> |
|
null |
||
-XX:[+/-]CNCEnableRemoteCompiler |
|
|||
-XX:ProfileName=<value> |
Name of the profile to identify the application and its version, for example, The ProfileName may only contain alphanumeric characters, and |
null |
||
-XX:CNCEngineUploadAddress=<host:port> |
Address to upload the compiler engine. Only needed when your Optimizer Hub has non-standard ports.
|
|||
-XX:[+/-]CNCStartWithLocalCompiler |
Start the JVM with the local compiler and disables it when the connection to Cloud Native Compiler succeeds. |
true |
||
-XX:CNCLocalFallbackOptLevelLimit=<3,2,1,0> |
Limit the OptLevel for local compilations when you enable Cloud Native Compiler. For more information, check the Zing documentation > Falcon Compiler Options > -XX:FalconOptimizationLevel. |
|
||
-XX:[+/-]CNCLocalFallbackProductivityBasedTriggering |
Enables and disables Local Fallback based on productivity checks. Other existing local fallback mechanisms like connection loss and server circuitbreaker are not controlled by this flag. Doesn’t impact telemetry collection or telemetry reporting. |
|
||
-XX:CNCProductivityLocalCompilationRatePerCompilerThread=<value> |
The number of compilations per second that a local compiler thread is presumed to be able to deliver when local productivity is being estimated for local fallback triggering purposes. The default value is derived from |
|
||
-XX:CNCProductivityMinEvaluationWindowSeconds=<value> |
The minimal size of the time window in seconds for productivity evaluation. |
|
||
-XX:CNCProductivityMinOutstandingCompilesForEvaluation=<value> |
Determines if there is enough work on the server to be able to trigger local fallback. |
|
||
-XX:[+/-]OptHubUseSSL |
Instructs the Zing JVM to communicate directly with Optimizer Hub without using SSL. Use this option if you installed Optimizer Hub without SSL. |
|
||
-XX:OptHubSSLRootsPath=<path to cert.pem> |
Instructs the Zing JVM to use and trust a specified SSL certificate on the filesystem. |
|||
-Xlog:[+/-]concomp |
Display messages describing communication with Optimizer Hub. |
|