JVM in a container: heap sizing, GC choice, async-profiler

What it is

The JVM was designed to own a machine. In a container it owns a cgroup, and three of its defaults are computed from what it believes the machine is:

Default max heap:      1/4 of "available memory"
Default GC:            chosen from "available processors" and memory
Default thread pools:  ForkJoinPool.commonPool, GC threads, JIT compiler
                       threads, all sized from "available processors"

Modern JVMs (10+, and backported to 8u191) read cgroup limits, so Runtime.availableProcessors() returns a value derived from the CPU limit and maxMemory() from the memory limit. That fixed the catastrophic version of the problem.

What remains is subtler and still costs money: heap is a fraction of the limit, not the limit, so a container with a 4 GB limit gets a 1 GB heap by default and 3 GB sits unused; and the CPU-count derivation is a ceil of the quota, so a limit of 1.5 CPUs reports 2 and a limit of 500m reports 1, which changes the GC selection.

What this is confused with: "the JVM is container-aware now, so it is fine." Container awareness means it reads the limits. It does not mean the defaults derived from those limits suit your workload, and the gap between a 25 percent heap default and a sensible 70 percent is most of the memory you are paying for.

The problem it solves

Before 8u191, the JVM read /proc/meminfo and /proc/cpuinfo, which report the host, not the cgroup:

Container limit:        2 GB
Host memory:            256 GB
JVM default max heap:   64 GB      <- 1/4 of the HOST
Result:                 the JVM grows past 2 GB and is OOMKilled with
                        no OutOfMemoryError, because the kernel killed it
                        before the JVM noticed

Exit 137 with no OutOfMemoryError in the logs is the signature, and it is still seen on Java 8 images that predate the backport.

The current problems, on a container-aware JVM:

Wasted memory. MaxRAMPercentage defaults to 25, so three quarters of a container's memory limit is unavailable to the heap. Some of that is legitimately needed for non-heap usage; 75 percent is far more than needed.

Non-heap memory that nobody counts. The container limit must cover heap plus metaspace plus code cache plus thread stacks plus direct buffers plus GC structures plus the JVM itself. A team that sets -Xmx equal to the container limit will be OOMKilled, and the error will be the kernel's rather than the JVM's, so it looks like a leak.

GC selection by machine class. The JVM picks SerialGC below roughly 2 CPUs and 1792 MB, and G1 above it. A container limited to 1 CPU gets SerialGC, which stops the world for every collection, and that is a surprising choice for a service.

Mechanics

Sizing the container, not just the heap

Container memory limit
├── Heap                    -Xmx / MaxRAMPercentage
├── Metaspace               class metadata; grows with classes loaded
├── Code cache              JIT-compiled code (default max 240 MB)
├── Thread stacks           threads x -Xss (default 1 MB on 64-bit)
├── Direct byte buffers     NIO, Netty; -XX:MaxDirectMemorySize
├── GC overhead             card tables, remembered sets: ~5-10% of heap for G1
├── Compressed class space  ~1 GB reserved, much less committed
└── JVM itself + malloc     ~50-100 MB

A worked sizing for a 4 GB container:
  Heap (-Xmx):              2560 MB    (62.5%)
  Metaspace:                 256 MB
  Code cache:                240 MB
  Thread stacks (200 x 1MB): 200 MB
  Direct buffers:            256 MB
  GC overhead (~8% of heap): 205 MB
  JVM + malloc:              100 MB
                            ─────────
                            3817 MB    -> fits in 4096 with headroom

Native Memory Tracking is how you replace those estimates with measurements:

$ java -XX:NativeMemoryTracking=summary ...
$ jcmd 1 VM.native_memory summary
Total: reserved=5242880KB, committed=3891200KB
-                 Java Heap (reserved=2621440KB, committed=2621440KB)
-                     Class (reserved=1114112KB, committed=  81920KB)
-                    Thread (reserved= 206848KB, committed= 206848KB)
                            (thread #201)
-                      Code (reserved= 253952KB, committed= 122880KB)
-                        GC (reserved= 198656KB, committed= 198656KB)

Thread (thread #201) at 206 MB is where runaway thread counts show up, and it is the single most common source of "the heap is fine and the container is OOMKilled." See the memory page's worked example, where 812 threads was 812 MB.

Set the percentage, not the absolute value:

# Portable across container sizes; survives someone changing the limit.
-XX:MaxRAMPercentage=65.0
-XX:InitialRAMPercentage=65.0     # avoid heap resizing during warmup
-XX:MinRAMPercentage=65.0         # applies BELOW 96MB; badly named

MinRAMPercentage does not mean a minimum heap. It is the percentage used when the container has less than about 96 MB, which is one of the worse names in the JVM. Setting Initial equal to Max avoids heap growth during startup, which matters because heap resizing during warmup causes GC pauses at exactly the wrong time.

The CPU count derivation, and why 1.5 is a bad limit

availableProcessors() = ceil(cpu.max quota / period)

CPU limit 500m  -> ceil(0.5) = 1
CPU limit 1     -> 1
CPU limit 1.5   -> ceil(1.5) = 2
CPU limit 2     -> 2

Fractional CPU limits round up, so the JVM sizes its thread pools for 2 CPUs while the cgroup permits 1.5, and every pool is over-provisioned relative to the quota. Combined with CFS throttling (see CPU limits), that produces GC threads competing for a quota they can exhaust in a fraction of a period.

# If you must use a fractional limit, tell the JVM the truth:
-XX:ActiveProcessorCount=2

# And size the pools explicitly rather than letting them derive:
-XX:ParallelGCThreads=2
-XX:ConcGCThreads=1
-Djava.util.concurrent.ForkJoinPool.common.parallelism=2

ForkJoinPool.commonPool is the one people forget, and it is what parallel streams use. A parallel stream on a container that reports 8 CPUs creates 7 worker threads regardless of the quota.

GC selection

Default selection:
  < 2 CPUs OR < 1792 MB    -> SerialGC
  otherwise                -> G1GC
GCPause targetHeap rangeRight for
SerialGCFull STW, proportional to heap< 500 MBTiny containers, batch
ParallelGCSTW, parallelanyThroughput batch work
G1GC~200 ms default target2 GB to 100+ GBThe general default
ZGC< 1 ms, heap-size independent8 GB to 16 TBLatency-critical, large heap
ShenandoahGC< 10 ms4 GB+Latency-critical, smaller heaps
# Latency-sensitive service, generational ZGC (JDK 21+):
-XX:+UseZGC -XX:+ZGenerational
-XX:MaxRAMPercentage=70

# Throughput batch job:
-XX:+UseParallelGC -XX:MaxRAMPercentage=80

# The general case:
-XX:+UseG1GC -XX:MaxGCPauseMillis=100 -XX:MaxRAMPercentage=65

Generational ZGC (JDK 21) is the change that made ZGC a default candidate. Non-generational ZGC had sub-millisecond pauses and needed substantially more heap headroom and more CPU, because it scanned the whole heap on every cycle. The generational version collects young objects separately, which cuts both.

The trade to state plainly: ZGC and Shenandoah spend CPU to avoid pauses. Concurrent collection means GC work happens alongside application threads, so throughput is typically 5 to 15 percent lower than ParallelGC. For a batch job that is a pure loss; for a service with a latency SLO it is the entire point.

Profiling: async-profiler

jstack and most JVM profilers use AsyncGetCallTrace at safepoints, which biases samples toward code that reaches a safepoint quickly and misses long-running loops entirely. That is the safepoint bias problem, and it means a profiler can confidently point at the wrong method.

async-profiler samples with perf_events and does not require a safepoint:

# CPU profile, 30 seconds, flame graph output.
./profiler.sh -d 30 -e cpu -f /tmp/cpu.html 1

# ALLOCATION profile: which call sites allocate, which drives GC pressure.
./profiler.sh -d 30 -e alloc -f /tmp/alloc.html 1

# Lock contention.
./profiler.sh -d 30 -e lock -f /tmp/lock.html 1

# Wall clock, which is what you want for latency: it samples ALL threads
# including blocked ones, so time waiting on I/O is visible.
./profiler.sh -d 30 -e wall -t -f /tmp/wall.html 1

-e wall is the mode people do not know about and usually need. CPU profiling shows where CPU is spent, and a service whose latency problem is waiting on a database shows almost nothing in a CPU profile. Wall-clock profiling shows where time is spent, including blocked threads, which is what a latency investigation needs.

Running it in a container requires two things:

securityContext:
  capabilities:
    add: ["SYS_ADMIN"]        # for perf_events; OR set the sysctl below

# Preferred: no capability needed, set on the node.
sysctl -w kernel.perf_event_paranoid=1
sysctl -w kernel.kptr_restrict=0

Adding SYS_ADMIN to profile is a bad trade given the container security page's argument; the node sysctl is the right approach, and it is why profiling access is usually a platform decision rather than an application one.

JDK Flight Recorder is the always-on alternative, at roughly 1 percent overhead:

-XX:StartFlightRecording=settings=profile,filename=/tmp/rec.jfr,maxsize=200M

JFR for continuous low-overhead recording, async-profiler for a deep dive, is the pairing that works.

A worked example: 6 GB containers running a 1.5 GB heap

A payments platform, 60 Java services, Kubernetes, JDK 17.

Baseline:

container memory limit:       6 GB (uniform, copied across all services)
JVM flags:                    -Xms512m -Xmx4g       <- absolute, and inconsistent
                              with the limit
actual heap used (p99):       1.4 GB
GC:                           G1 (default)
p99 latency (flagship):       340 ms
GC pause p99:                 180 ms
OOMKills:                     ~40/month
nodes:                        88

Three separate problems, and the OOM kills were the entry point.

Problem 1: -Xmx4g in a 6 GB container with unmeasured non-heap usage.

$ jcmd 1 VM.native_memory summary
Total: committed=5734400KB
-  Java Heap (committed=4194304KB)     # 4.0 GB
-      Class (committed=  204800KB)
-     Thread (committed=  614400KB)     # 600 MB: 600 threads
-       Code (committed=  245760KB)
-         GC (committed=  335872KB)
-      Other (committed=  139264KB)

5.73 GB committed against a 6 GB limit, so any spike in direct buffers or thread count crossed it. The kernel killed the process; the JVM never saw an OutOfMemoryError, so the logs showed a clean shutdown followed by a restart.

-> the 600 threads were a Tomcat maxThreads of 200 plus three separate
   HTTP client pools each defaulting to 200

Problem 2: the heap was four times larger than needed. p99 heap usage was 1.4 GB against a 4 GB -Xmx, and the 6 GB container limit was uniform across 60 services regardless of what they did.

Problem 3: G1 with a 4 GB heap and a 180 ms pause p99 on a service with a 200 ms latency SLO. G1's default MaxGCPauseMillis is 200, so it was meeting its own target and that target consumed the entire budget.

The changes:

# 1. Percentage-based, so it tracks whatever the limit becomes.
-XX:MaxRAMPercentage=70
-XX:InitialRAMPercentage=70

# 2. Bound the non-heap explicitly rather than discovering it.
-XX:MaxMetaspaceSize=256m
-XX:ReservedCodeCacheSize=240m
-XX:MaxDirectMemorySize=256m
-Xss512k                            # 200 threads x 512k = 100MB, not 200MB

# 3. Generational ZGC for the latency-critical services.
-XX:+UseZGC -XX:+ZGenerational

// 4. Bound the thread pools, which was the actual OOM cause.
server.tomcat.threads.max=64        // was 200
// and one shared HTTP client pool instead of three:
PoolingHttpClientConnectionManager cm = new PoolingHttpClientConnectionManager();
cm.setMaxTotal(200);
cm.setDefaultMaxPerRoute(32);

And the sizing done per service rather than uniformly:

# Per service, from 14 days of JFR and container metrics.
heap_needed = p99(jvm_memory_used_bytes{area="heap"}) * 1.5
nonheap     = measured_from_nmt()
limit       = round_up(heap_needed / 0.70 + nonheap_headroom)

             before (uniform)   after (measured)
service A    6 GB               2 GB
service B    6 GB               3 GB
service C    6 GB               8 GB    <- one service was UNDER-provisioned
...
mean         6 GB               2.9 GB

One service was under-provisioned, and the uniform limit had hidden it: it was OOM-killed regularly and the team had assumed it leaked.

Results:

                              before      after
container limit (mean)        6 GB        2.9 GB
heap (mean)                   4 GB        2.0 GB (70% of limit)
committed vs limit            96%         74%
OOMKills                      ~40/mo      0
threads (flagship)            600         96
GC pause p99                  180 ms      0.8 ms      (ZGC)
p99 latency (flagship)        340 ms      121 ms      (-64%)
throughput                    baseline    -6%         (ZGC's CPU cost)
nodes                         88          51          (-42%)

Forty-two percent fewer nodes and a 64 percent p99 reduction, and throughput dropped 6 percent, which is the ZGC trade paid deliberately.

The profiling finding, which came after. With GC pauses gone, the remaining p99 was investigated with async-profiler in wall-clock mode:

./profiler.sh -d 60 -e wall -t -f /tmp/wall.html 1

Wall-clock profile, p99 requests:
  47%  socketRead0            <- waiting on a downstream service
  18%  ObjectMapper.readValue <- JSON deserialisation
  11%  regex Pattern.matcher  <- a validation regex compiled PER REQUEST
   9%  ...

A CPU profile of the same service showed Pattern.compile at 34 percent and almost nothing about the socket read, because a blocked thread uses no CPU. The CPU profile would have sent them to optimise JSON parsing; the wall-clock profile showed that half the time was waiting on a downstream call.

// The regex fix was still worth it: 11% of wall time, and trivial.
private static final Pattern VALID = Pattern.compile("^[A-Z]{2}\\d{6}$");  // static

p99 latency:  121 ms -> 96 ms

Final:

                              before      after
p99 latency                   340 ms      96 ms       (-72%)
GC pause p99                  180 ms      0.8 ms
OOMKills                      ~40/mo      0
nodes                         88          51
container limit (mean)        6 GB        2.9 GB

The transferable findings are two. MaxRAMPercentage rather than -Xmx, because it tracks the limit and cannot drift out of sync with it, and because the default of 25 percent is far below what a service should use. And wall-clock profiling rather than CPU profiling for latency work, because a service waiting on I/O shows almost nothing in a CPU profile and the CPU profile will confidently point somewhere else.

Production evidence

Container awareness landed in JDK 10 (JDK-8146115) and was backported to 8u191. Before that the JVM read host /proc, and the "OOMKilled with no OutOfMemoryError" signature is the canonical symptom on older images. UseContainerSupport is on by default and can be disabled, which is occasionally the cause of a regression on a JVM upgrade.

Generational ZGC shipped in JDK 21 (JEP 439) and is the change that made ZGC viable as a general default: the non-generational version required substantially more headroom and CPU because it scanned the whole heap every cycle.

async-profiler is widely used precisely because it avoids the safepoint bias in AsyncGetCallTrace-based profilers. Its wall-clock mode is the feature that distinguishes it for latency work, and the safepoint-bias problem was documented by Nitsan Wakart and others.

JDK Flight Recorder was open-sourced in JDK 11 and is designed for always-on production recording at roughly 1 percent overhead, which is why the JFR-plus-async-profiler pairing has become standard.

Spring Boot's and Micronaut's container guidance both recommend MaxRAMPercentage over -Xmx, and Paketo and Google's Java buildpacks set it automatically along with a computed thread-stack budget, which is an implicit acknowledgement that the defaults do not suit containers.

kernel.perf_event_paranoid being the blocker for profiling in containers is documented by async-profiler, and the recommendation to set the node sysctl rather than granting SYS_ADMIN is theirs.

The debate

-Xmx or MaxRAMPercentage? Percentage, essentially always. It tracks the container limit, so a limit change does not silently leave the heap wrong, and it survives the same image running in differently-sized containers. The default of 25 percent is the problem, not the mechanism: 65 to 75 percent is a reasonable range for a service whose non-heap usage you have measured with Native Memory Tracking.

Which GC? G1 as the default, ParallelGC for batch throughput, generational ZGC when you have a latency SLO that GC pauses would consume. The trade is explicit: concurrent collectors spend 5 to 15 percent throughput to remove pauses, so for a batch job it is a pure loss and for a latency-sensitive service it is the entire point. SerialGC is what you get by accident on small containers and is rarely what you want for a service.

Is a container-aware JVM enough? No, and this is the position worth holding. It reads the limits, which prevents the catastrophic version. The defaults derived from those limits (25 percent heap, GC chosen by machine class, thread pools sized by a rounded-up CPU count) still need attention, and the gap between a 25 percent default and a measured 70 percent is most of the memory bill.

Should you use fractional CPU limits? Prefer not to, because availableProcessors() rounds up, so a 1.5-CPU limit reports 2 and every derived pool is sized for capacity the cgroup will not grant. If you must, set ActiveProcessorCount explicitly. And the broader argument from the requests and limits page applies: for most services, do not set a CPU limit at all.

CPU profile or wall-clock profile? Wall clock for latency, CPU for throughput and cost. The distinction is that a blocked thread consumes no CPU, so a service whose p99 is dominated by a slow downstream call shows almost nothing relevant in a CPU profile. In the worked example the CPU profile pointed at regex compilation at 34 percent while the wall-clock profile showed 47 percent waiting on a socket, and only one of those was the latency problem.

Is always-on profiling worth it? JFR at roughly 1 percent overhead, yes, and it is the difference between investigating an incident with data and reproducing it first. The constraint is usually the security posture rather than the overhead: async-profiler needs perf_events, which means a node sysctl or a capability, and granting SYS_ADMIN to profile is a worse trade than setting kernel.perf_event_paranoid on the node.

Follow-up Q&A

"Why was the JVM OOMKilled with no OutOfMemoryError?"

Because the kernel killed the process before the JVM reached its own heap limit. That happens when the container limit must cover heap plus metaspace plus code cache plus thread stacks plus direct buffers plus GC structures, and only the heap was budgeted. On pre-8u191 JVMs it also happens because the JVM read host /proc and sized its heap at a quarter of the host's memory. The diagnostic is jcmd VM.native_memory summary with Native Memory Tracking enabled, and the usual culprit is the thread count.

"Why MaxRAMPercentage instead of -Xmx?"

Because it tracks the container limit, so changing the limit cannot leave the heap silently wrong, and the same image works in differently-sized containers. The default of 25 percent is the thing to change: it leaves three quarters of the limit unavailable to the heap, which is far more headroom than non-heap usage needs. Measure non-heap with NMT and set the percentage from that, typically 65 to 75.

"How does the JVM decide how many CPUs it has, and why does it matter?"

ceil(quota / period) from the cgroup, so a 1.5-CPU limit reports 2 and a 500m limit reports

  1. It matters because GC thread count, JIT compiler threads and ForkJoinPool.commonPool.parallelism all derive from it, so fractional limits over-provision every pool relative to what the cgroup will grant, and combined with CFS throttling the GC threads can exhaust the quota in a fraction of a period. Set ActiveProcessorCount explicitly if you use fractional limits.

"Which garbage collector would you choose?"

G1 as the general default. ParallelGC for batch work where throughput is the only thing that matters. Generational ZGC (JDK 21+) when there is a latency SLO that GC pauses would eat: in one case it took GC pause p99 from 180 ms to 0.8 ms on a service with a 200 ms budget. The cost is explicit, 5 to 15 percent throughput, because concurrent collection does GC work alongside application threads. For batch that is a pure loss; for a latency-sensitive service it is the point.

"CPU profile or wall-clock profile?"

Wall clock for latency work. A blocked thread consumes no CPU, so a service whose p99 is dominated by waiting on a downstream call shows almost nothing relevant in a CPU profile. In one case the CPU profile put regex compilation at 34 percent and the wall-clock profile showed 47 percent in socketRead0, waiting on a downstream service. Both findings were real; only one was the latency problem, and the CPU profile would have sent the team to optimise the wrong thing.

"How do you profile a JVM in a container?"

async-profiler needs perf_events, which means either SYS_ADMIN on the container or kernel.perf_event_paranoid=1 on the node. The node sysctl is the right answer, because granting SYS_ADMIN undoes the capability hardening for the sake of a profile. Pair it with JFR running continuously at about 1 percent overhead, so an incident can be investigated from recorded data rather than reproduced first.

Common misconceptions

"Modern JVMs are container-aware, so the defaults are fine." They read the limits, which prevents the catastrophic failure. The defaults derived from those limits (25 percent heap, GC by machine class, pools sized from a rounded-up CPU count) still need setting.

"Set -Xmx to the container limit." The limit must also cover metaspace, code cache, thread stacks, direct buffers and GC structures. Setting the heap to the limit guarantees a kernel OOM kill, and it will not produce an OutOfMemoryError.

"MinRAMPercentage sets a minimum heap." It is the percentage used when the container has less than about 96 MB. InitialRAMPercentage is the starting heap size.

"ZGC is strictly better." It trades 5 to 15 percent throughput for sub-millisecond pauses. For batch work that is a loss with no benefit.

"A CPU profile shows where the time goes." It shows where CPU goes. Time spent blocked on I/O is invisible in it, which for a latency investigation is usually most of the time.

Interview delivery note

Say this verbatim: "The container limit has to cover heap plus metaspace plus code cache plus thread stacks plus direct buffers plus GC structures, so -Xmx equal to the limit is a guaranteed kernel OOM kill with no OutOfMemoryError in the logs. I use MaxRAMPercentage around 70 after measuring non-heap with Native Memory Tracking, because the default of 25 leaves three quarters of the memory you are paying for unused." The failure signature and the corrected default with its justification.

The senior-versus-staff separator is wall-clock profiling for latency. A senior engineer profiles the service and optimises what the profile shows. A staff engineer knows that a blocked thread uses no CPU, so a CPU profile of a service waiting on a downstream call will confidently point somewhere else, and uses -e wall instead: in one case the CPU profile said regex compilation at 34 percent and the wall-clock profile said 47 percent waiting on a socket. Both were real; only one was the latency.

The second signal is jcmd VM.native_memory summary as the first step for an OOM kill, specifically to check the thread count. "Six hundred threads at 1 MB each is 600 MB of stacks that nobody budgeted" is a five-second diagnosis for a failure that otherwise looks like a leak.

Further reading

  • JDK-8146115 and the UseContainerSupport documentation, for what container awareness does and does not do.
  • JEP 439 (Generational ZGC), for the design and the reason non-generational ZGC needed more headroom.
  • async-profiler's documentation, particularly the wall-clock mode and the perf_event_paranoid requirement.
  • The Native Memory Tracking documentation (jcmd VM.native_memory), for measuring the non-heap budget rather than estimating it.