About Cloud Native Compiler
Cloud Native Compiler is a component of Optimizer Hub that provides a server-side optimization solution that offloads JIT compilation to separate and dedicated service resources, providing more processing power to JIT compilation while freeing your client JVMs from the burden of doing JIT compilation locally.
JIT Optimization
JIT optimization provides a multitude of benefits, including the ability to use speculative optimizations that lead to faster eventual code. However, traditional on-JVM JIT compilers must share the JVMs local CPU resources and compete with the application logic in using that capacity. This presents several challenges:
-
Optimization limitations:
-
The JIT compiler is limited in resources. Resulting optimizations take time to arrive at, and benefits are limited by the practical amount of time that applications can wait for optimization and warm-up to complete.
-
The JIT compiler is limited in how aggressively it can afford to optimize code. The resulting optimizations are not as fast as they could be if the optimizer had more resources available.
-
-
Application performance limitations during warm-up:
-
Optimization takes time to complete, and application code runs significantly slower and less efficiently until the JIT compilers optimize it.
-
JIT compilation work competes with the application for resources. Not all CPU and memory resources are devoted to application threads.
-
-
Resource allocation and utilization:
-
Resources (CPU and memory capacity) used for JIT optimization are only needed and utilized during warm-up, which is a fraction of the overall lifetime of each Java process. Instances must reserve (and customers pay for) these under-utilized resources for the duration of the run of each application instance.
-
Falcon JIT
Azul Zulu Prime Builds of OpenJDK replace OpenJDK’s C2 JIT compiler with the Falcon JIT compiler. The Falcon JIT compiler can run different levels of optimizations, and its upper tier of optimizations produces optimized code that can run significantly faster than code produced by the OpenJDK C2 compiler.
Using more aggressive optimization levels requires more resources, and when using JVM-local JIT compilers for optimization, resource tradeoffs can often lead to a choice of lowering optimization levels in favor of improved warmup times. Cloud Native Compiler eliminates these tradeoffs by removing JIT compilation work from individual JVMs, and shifting the work of the Falcon JIT compiler to a separate shared service. This shift of work and associated resources allows the Cloud Native Compiler to apply even the most aggressive Falcon JIT optimization levels without disrupting individual JVM behavior. The Cloud Native Compiler can bring to bear practically unlimited Falcon JIT compilation resources when a JVM needs them, and later scale those resources down when they are unused and unneeded. This results in JVMs that can consistently serve higher amounts of traffic in smaller footprint.