Question: how does this affect JVM ergonomics?

[tschut]

On startup, a JVM will typically determine certain behavior (e.g. thread pool size, garbage collector defaults) based on available system resources (CPU, memory). On top of that, so may application code. Typically the amount of CPUs available is used to determine basic concurrency in some processes. Also see the Oracle docs on ergonomics.

My question is: how does applying kube-startup-cpu-boost influence this? Will the JVM (and application code) detect the "boosted" CPU number? This may have adverse effects on performance when CPU is reverted to the request settings later.

I realize I can test this to see how it works in practice (and hope to find some time for that), but this seems pretty fundamental and I'd love to see the behavior as part of the documentation/specification, especially since this project explicitly targets JVM applications.

[mikouaj]

Hi @tschut ,
Thanks you for rising this, thats a very valid question. The answer is yes, the CPU boost operation may affect the performance of the GC and may also affect the performance of the concurrency in the applications. The number of active processors count is not updated dynamically, just like the GC thread pools (even in modern GCs like G1GC or Shenandoah). Unfortunately both are set once during the startup.

The performance impact may vary, depending on different factors like:

• Size of the application, smaller and more optimized code may result in GC kicking-in less likely
• Traffic patterns as busts they may result in more often garbage collections
• How many CPUs were added during boost operation - it should be limited as much as possible

What causes the performance impact? Running more threads than CPU cores will result more context switching wasting some CPU cycles that could be allocated to do the real work. Boosting CPU for JVM apps is a tradeoff.

The kube-startup-cpu-boot should be treated as a workaround when other options are not possible:

• Modernizing the app to compile into native byte-code as running JVM in a container is questionable
• Removing CPU limits so that container will always get max number of CPUs available on a node (Burstable QoS class of a POD)

[tschut]

Thanks for your response @mikouaj.

Is this something you could consider adding to the documentation? It's not mentioned anywhere last I checked, and it may cause fundamentally unexpected behavior in applications at runtime.

Also, I'm wondering about potential solutions to this. The best I've come up with so far is a configurable delay on the boost, which could be used to give the JVM time to determine ergonomics with the end-state of resources before the boost is applied. Of course, finding the right value for this delay might be tricky, but at least it offers a possibility.

Another thought (but my experience is limited here, I can't say how feasible this is) is that it would perhaps make sense to have sort of "readiness" probe that determines if the boost can be applied. This would make it possible for a java application to export something in the environment, which the "boostable"-probe could then check. If this export happens very early in the application (but after JVM ergonomics have been determined), considerable gains for applications with long startup times might still be possible.

Btw,

running JVM in a container is questionable

I don't agree with this statement. Running code in a (J)VM has considerable advantages in certain quite common cases, and a warmed up JVM can be faster than native bytecode.

[mikouaj]

Is this something you could consider adding to the documentation?

Yes thats a good idea, will consider this.

it may cause fundamentally unexpected behavior in application runtime

I do not agree its a fundamentally unexpected behavior. The application will run same way and what changes is under the hood - the waste of some CPU cycles due to scheduling. Note that scheduling is OS dependent anyway. Packing JVM into container and running it on a dynamic cluster platform like Kubernetes means that CPU scheduling will happen anyway, just like container restarts - in a contrast to JVM running on a steady application servers.

running JVM in a container is questionable

I don't agree with this statement. Running code in a (J)VM has considerable advantages in certain quite common cases, and a warmed up JVM can be faster than native bytecode.

I agree that JVM has advantages but not in a context of running it inside a container. You may treat it as my own opinion but this is virtualization inside virtualization. The one of the primary goals of JVM was portability which is no longer needed when running in a container - the container provides portability. Another aspect is being able to scale out and in quickly - the JVM was not designed for that and this is desired when running in Kubernetes, especially in a cloud.

Also, I'm wondering about potential solutions to this.

One solution would be... not running JVMs on a cluster platforms like Kubernetes. Another is to make JVMs evolve and adapt to platforms that provide dynamic resource allocation.

The best I've come up with so far is a configurable delay on the boost,

Thats not a bad idea but the overall boost is beneficial to speed up JVM optimizations and, class loading etc what happens immediately after start. Later on, the benefits of CPU boosting may be neglectable.