Solving Java Warmup Issues in Kubernetes: Best Practices

The first few requests to a freshly deployed Java service are often the slowest, and in Kubernetes they can be slow enough to time out. At Xflow, early calls to a Java application were taking close to 10 seconds and returning 5xx errors on every deploy.

We traced it to the JVM's warmup behaviour colliding with our Kubernetes CPU limits. Solving it is the kind of reliability work our platform and developers teams do day to day.

This post explains why Java is slow to warm up on Kubernetes, how we diagnosed it, and the fixes that cut first-request latency from about 10 seconds to 750 milliseconds and pod startup from around 90 seconds to 30.

The fixes below are what our platform team shipped, with the same rigour we bring to reliability work across the stack. Here is the short version:

• Root cause: JIT compilation is CPU-heavy at startup, and Kubernetes CFS CPU limits throttle that burst.
• Biggest lever: give the JVM CPU headroom with Burstable QoS, meaning the limit is set higher than the request.
• Application side: pre-establish database connections and run no-op queries during startup.
• JVM side: use -XX:TieredStopAtLevel=3 and bound the compiler threads when CPU is tight.
• Safety net: add a startup probe so a slow start is not killed mid-warmup.
• Advanced: evaluate AppCDS or CRaC for near-instant starts.
• Our result: first-request latency fell from ~10s to ~750ms, and startup from ~90s to ~30s.

Two things combine to make a Java pod sluggish right after it starts: how the JVM reaches peak speed, and how Kubernetes rations CPU.

The JVM warms up through JIT compilation. It starts by interpreting bytecode, then uses Just-In-Time (JIT) compilation to turn hot methods into native code.

This runs in tiers. The C1 (client) compiler produces fast, lightly optimised code early, and the C2 (server) compiler later recompiles the hottest methods with heavier optimisation.

Until those hot paths are compiled, the service runs slower. The compilation itself is CPU-intensive, running on background threads while your app is also loading classes and serving traffic.

Kubernetes CPU limits throttle that burst. A CPU limit is a time quota enforced by the Linux kernel's Completely Fair Scheduler (CFS). The kernel splits time into 100ms windows, so a limit of 500m grants 50ms of runtime per window.

Once the quota is spent, the kernel freezes every thread in the container until the next window opens. During warmup the JIT compiler alone can exhaust a small quota, so the JVM is paused at the exact moment it needs CPU most.

That is the warmup problem in one line: a compilation storm meets a tight CPU quota, and the kernel keeps hitting pause.

The symptom was consistent. Right after a deployment the first calls timed out or returned 5xx, then everything settled. We worked through the layers rather than guessing.

First we looked at the application. The service opened database connections lazily on the first request, so we pre-established them at startup. That removed about 100ms, confirming connection setup was a cost but not the main one.

Next we primed the data path with simple no-op queries during startup. That bought a 3 to 4 second improvement, yet the first calls still took around 6 seconds.

The remaining delay pointed at infrastructure, where compute cost and performance meet. Pod metrics showed heavy CPU throttling during startup. The JVM was being paused by the kernel, not by our code.

1. Prime connections and queries at startup

Move expensive one-time work out of the first user request. Pre-establish database and search connections during bootstrap so the first call does not wait on a cold pool.

Then run one lightweight query per connection to warm the driver and data path. A plain SELECT 1 against MySQL, or a trivial match-all against OpenSearch, is enough.

In our case this priming removed roughly 100ms from connection setup and another 3 to 4 seconds from the first real query. It is not a full fix on its own, but it clears the cold-start costs so the real bottleneck stands out.

2. Give the JVM CPU headroom during startup

This was our biggest lever. A JVM needs far more CPU while loading classes and running the JIT compiler than it does in steady state, so sizing the limit for steady state guarantees throttling at boot.

As a test we raised CPU from 0.896 to 4 for both request and limit. First-request latency dropped under 750ms and pod startup fell from about 90 seconds to 30.

The lesson is simple: size the CPU limit for the startup peak, not the steady-state average. The throttling, not the application code, had been the dominant cost all along.

3. Use Burstable QoS so the boost is not wasted

Pinning request and limit both to 4 gives every pod a large permanent reservation it needs for only a few seconds. Kubernetes assigns Guaranteed QoS when request equals limit, and Burstable QoS when the limit is higher than the request.

We moved to Burstable: request stays at the steady-state value, limit set to 2 after testing values between 2 and 4. The pod bursts into spare node CPU during warmup, then settles, without reserving four cores around the clock.

resources:
  requests:
    cpu: "896m"   # steady-state need
  limits:
    cpu: "2"       # headroom to absorb the JIT compilation burst

One caveat: bursting only works if the node actually has spare CPU, so leave slack on nodes that run warmup-heavy pods.

4. Tune JIT compilation for faster warmup

If extra CPU alone is not enough, tell the JVM to do less compilation up front. Stopping tiered compilation at C1 skips the expensive C2 pass.

That typically warms up around 50% faster in exchange for roughly 5% lower peak throughput, a trade many request-serving services will take. On small pods, also cap the compiler threads so JIT does not crowd out request handling.

# Skip C2 for faster warmup, at some peak-throughput cost
-XX:TieredStopAtLevel=3

# Bound JIT compiler threads on small containers (e.g. a 2-CPU pod)
-XX:CICompilerCount=2

For example, a Spring Boot API on a 2-CPU pod that only needs to serve steady traffic often runs fine at TieredStopAtLevel=3, trading a little peak speed for a much faster, throttle-free start. We kept full JIT, but these flags are the first thing to reach for when headroom is scarce.

5. Protect slow starts with a startup probe

A readiness probe that fires before the JVM is warm marks the pod unready, and an aggressive liveness probe can kill it mid-warmup, causing a restart loop.

Use a startup probe to give the container time to warm up, and only let liveness take over once it passes. The example below allows up to about 150 seconds (30 failures at 5 second intervals) before liveness applies.

startupProbe:
  httpGet: { path: /healthz, port: 8080 }
  failureThreshold: 30
  periodSeconds: 5

Set failureThreshold from your measured worst-case startup, not an optimistic one, or a slow node will still trigger restarts.

6. Consider AppCDS and CRaC for near-instant starts

For deeper cuts, the OpenJDK toolkit has grown. Application Class Data Sharing (AppCDS) archives the classes your app loads so later starts skip repeated loading, often cutting startup by up to half.

You build the archive once, for example with -XX:ArchiveClassesAtExit=app.jsa, then start with -XX:SharedArchiveFile=app.jsa. Coordinated Restore at Checkpoint (CRaC) goes further: warm the JVM once, snapshot its state, and restore from that snapshot so the process starts already warm.

Project Leyden is pursuing a similar goal through static images. These carry build and operational trade-offs, so treat them as the next step once probes, CPU headroom and JIT tuning are in place.

7. Right-size compute and watch burst credits

Burstable QoS relies on spare CPU on the node, so schedule with that in mind. If you run on AWS burstable instances such as t3a, the baseline is around 40%, and bursting draws on CPU credits that cost money once exhausted.

For example, a deploy that restarts many warmup-heavy pods at once can burn a day's credits in minutes. We added CPU credit-balance alerts so a heavy rollout does not quietly run up a bill, and so we know when a workload has outgrown a burstable instance family.

The win came from combining application priming with an infrastructure fix, not from a single flag. Priming removed a few seconds, but the step change was giving the JVM CPU headroom so the kernel stopped throttling it during the compilation storm.

• Java is slow at boot because JIT compilation is CPU-heavy, and Kubernetes CFS quotas throttle that burst.
• Size CPU limits for startup, not steady state, or use Burstable QoS to borrow spare CPU during warmup.
• Prime connections and run no-op queries so the first request does not pay cold-start costs.
• Reach for -XX:TieredStopAtLevel=3 and a bounded compiler-thread count when CPU is tight.
• Use a startup probe to stop premature restarts, and evaluate AppCDS or CRaC for near-instant starts.

Java warmup issues in Kubernetes are almost always a mismatch between the JVM's CPU-hungry startup and a CPU quota tuned for steady state.

Fix the mismatch first with headroom and Burstable QoS, prime the application to remove cold-start costs, then tune JIT and probes for the rest.

For a payments platform where a timed-out first request can mean a failed transaction, that reliability is not optional. If you build low-latency systems, see the open roles on our careers page.

More engineering blogs by Xflow:

• Cost Optimization Strategies from Xflow
• Xflow Enhances Email Security and Communications
• Debugging Memory Growth Issues