Requests, limits, QoS classes and eviction order

What it is

Two numbers per container per resource, meaning entirely different things:

Requests are what the scheduler uses. A pod requesting 500m CPU and 1Gi memory will only be placed on a node with that much unallocated. Requests are a claim on capacity and they are what the node's allocatable budget is spent on.

Limits are what the kernel enforces at runtime, through cgroups. They are a ceiling, and the two resources behave completely differently at the ceiling:

	Over the limit	Mechanism
CPU	Throttled	CFS quota: the process is descheduled until the next period
Memory	Killed	The cgroup OOM killer terminates the process, exit 137

CPU is compressible and memory is not, and that single asymmetry explains most of this page. You can give a process less CPU and it goes slower; you cannot give it less memory than it is using.

The QoS class is derived automatically from the relationship between requests and limits, and it determines eviction order:

Guaranteed:  requests == limits, for every resource, in every container
Burstable:   requests set, and less than limits (or limits unset)
BestEffort:  no requests and no limits at all

What this is confused with: requests as a reservation. A pod requesting 1 CPU is not given 1 CPU. It is scheduled as though it will use 1 CPU, and at runtime it can use as much as is free up to its limit. Requests bound what the scheduler commits; they do not bound what runs.

The problem it solves

Without requests, the scheduler is guessing. It has no idea how much of a node a pod will consume, so it packs by pod count, and a node running eight memory-hungry pods thrashes while another runs eight idle ones.

Without limits, one pod can take a node down. A memory leak in one container consumes the node's memory, the node-level OOM killer fires, and it kills by an oom_score heuristic that may well select something else entirely, including a system-critical pod.

And the failure mode people actually hit is neither of those. It is setting both badly, in one of two specific ways:

Requests too low. The scheduler over-packs the node because it believes the pods are small. Everything fits on paper and the node is genuinely oversubscribed, so pods throttle and get evicted under pressure. The symptom is latency that correlates with what else happens to land on the node, which looks like random flakiness.

CPU limits set at all. This is the contentious one and it has its own page (see CPU limits and CFS throttling). A CPU limit does not slow a process smoothly; it descheduies it for the remainder of a 100ms period, so a service using well under its limit on average can still see 100ms latency spikes when a burst exhausts the quota.

Mechanics

What the numbers do, mechanically

resources:
  requests: {cpu: "500m", memory: "1Gi"}
  limits:   {cpu: "2",    memory: "2Gi"}

CPU request becomes a cgroup weight:

cpu.weight (cgroups v2) = requests.cpu proportionally

It only matters under contention. If the node is not saturated, a container requesting 500m can use every core it can find. When the node is saturated, containers get CPU in proportion to their requests. The CPU request is a share of the contended remainder, not an allocation.

CPU limit becomes a hard quota:

cpu.max = "200000 100000"     # 2 CPU-seconds per 100ms period

Exhaust the quota inside a period and the process is descheduled until the next one. See the throttling page for why this bites even at low average utilisation.

Memory request is scheduling only. Nothing enforces it at runtime.

Memory limit becomes memory.max, and exceeding it invokes the cgroup OOM killer. Exit code 137, OOMKilled in the pod status, and no graceful shutdown.

The eviction order, which is what QoS is for

When a node comes under memory pressure (or disk pressure), the kubelet evicts pods to reclaim resources, in a defined order:

1. BestEffort pods                              (no requests: nothing promised)
2. Burstable pods USING MORE than their request (in excess-over-request order)
3. Burstable pods using less than their request
4. Guaranteed pods                              (last, and only if necessary)

Within a tier, the ordering is by how far a pod exceeds its memory request, then by priority. So a Burstable pod requesting 500Mi and using 3Gi is evicted before one requesting 2Gi and using 2.1Gi, even though the second is using more in absolute terms.

That rule is the practical reason to set memory requests near actual usage. A pod whose request is far below its real usage is at the front of the eviction queue permanently, and it presents as "this service gets restarted a lot and we don't know why."

oom_score_adj reflects the same ordering at the kernel level:

Guaranteed:   -997          (very unlikely to be chosen by the node OOM killer)
BestEffort:   1000          (chosen first)
Burstable:    2..999, computed from requests relative to node capacity

Allocatable, and where the node's capacity goes

Node capacity:           16 CPU, 64Gi
  - kube-reserved:       1 CPU, 2Gi      (kubelet, container runtime)
  - system-reserved:     0.5 CPU, 1Gi    (sshd, systemd, the OS)
  - eviction-threshold:  0.5Gi           (headroom before eviction starts)
                        ─────────────
Allocatable:             14.5 CPU, 60.5Gi   <- what the scheduler can commit

The scheduler commits against allocatable, and it commits requests, not limits. So a node can be 100 percent committed by requests and idle, or 40 percent committed and saturated, and both are normal. This is why "the cluster is 90 percent allocated" and "the cluster is 30 percent utilised" are simultaneously true and both meaningful.

Sizing, as arithmetic rather than intuition

Memory request: p99 of observed working set, plus ~20% headroom.
Memory limit:   request x 1.2-1.5, or EQUAL to the request for Guaranteed.
CPU request:    p50-p90 of observed usage. This is the scheduling claim.
CPU limit:      usually UNSET. See below.

# Memory: use the working set, not RSS, and take a high percentile over a long window.
quantile_over_time(0.99,
  container_memory_working_set_bytes{pod=~"web-.*"}[7d])

# CPU: the rate, at a percentile that reflects real bursts.
quantile_over_time(0.90,
  rate(container_cpu_usage_seconds_total{pod=~"web-.*"}[5m])[7d:5m])

container_memory_working_set_bytes, not container_memory_usage_bytes. The latter includes reclaimable page cache, so it drifts upward toward the limit and looks alarming when nothing is wrong. The working set is what the OOM killer actually considers, and using the wrong metric is the most common cause of a memory limit set two or three times higher than necessary.

The Guaranteed question

Setting requests == limits gets you Guaranteed QoS, which buys:

Last in the eviction order.
oom_score_adj of -997.
Exclusive CPUs, if the node's CPU manager policy is static and the CPU request is a whole number. That removes scheduler migration and gives cache locality, which for latency-sensitive workloads is a real and measurable gain.

It costs: no bursting. A Guaranteed pod cannot use idle capacity on the node, so the cluster is less efficiently packed and you pay for headroom that sits unused.

Guaranteed for memory is nearly always right for anything you care about, because memory is incompressible and the eviction protection is real. Guaranteed for CPU means setting a CPU limit, which brings the throttling problem, so it is right for latency-critical workloads on static CPU manager nodes and wrong for most services.

A worked example: 4,000 restarts a month, and none of them were leaks

A platform running about 340 services on 60 nodes. Restart rate was high enough to be background noise nobody investigated.

Baseline:

OOMKilled restarts:            ~2,800/month
evicted pods:                  ~1,200/month
node CPU utilisation (mean):   31%
node memory utilisation:       74%
cluster allocated by requests: 94% CPU, 88% memory
p99 latency, one flagship svc: 890ms
CPU throttling (mean, that svc): 18% of periods

Ninety-four percent allocated and 31 percent utilised is the shape that says requests are wrong. The cluster was full on paper and idle in practice, and nodes could not be bin-packed further.

An audit of resource specs across 340 services:

no requests or limits at all (BestEffort):      41 services
requests set, limits unset (Burstable):         88
requests == limits (Guaranteed):                26
requests set, limits set, requests << limits:  185
memory limit > 4x observed p99 working set:    142
CPU limit set:                                 211
copied from another service's manifest:        ~60% (by author admission)

Most specs had been copied. The most-copied manifest requested 100m CPU and 128Mi memory with limits of 2 CPU and 2Gi, and it had been pasted into services whose actual usage bore no relation to it.

Fix 1: right-size from observation, not intuition.

# Generated per service from 14 days of Prometheus data.
def recommend(service):
    mem_p99 = query(f'quantile_over_time(0.99, '
                    f'container_memory_working_set_bytes{{service="{service}"}}[14d])')
    cpu_p90 = query(f'quantile_over_time(0.90, '
                    f'rate(container_cpu_usage_seconds_total{{service="{service}"}}[5m])[14d:5m])')
    return {
        "requests": {"memory": round_up(mem_p99 * 1.2), "cpu": round_up(cpu_p90)},
        "limits":   {"memory": round_up(mem_p99 * 1.2)},   # == request: Guaranteed for memory
        # CPU limit deliberately omitted.
    }

                              before      after
cluster allocated (CPU)       94%         52%
cluster allocated (memory)    88%         71%
node CPU utilisation          31%         38%
nodes required                60          44        (-27%)

Twenty-seven percent fewer nodes, because accurate requests let the scheduler pack properly. The requests had been simultaneously too high (CPU, copied from a template) and too low (memory, for the services that were actually growing).

Fix 2: memory requests equal to limits.

OOMKilled restarts:    2,800/month -> 210/month
evicted pods:          1,200/month -> 40/month

Two mechanisms. The right-sizing removed the limits that were genuinely too low. And Guaranteed QoS for memory moved those pods to the back of the eviction queue, which eliminated most of the evictions: they had been Burstable pods using far more than their (copied, tiny) memory request, which put them at the front of the queue permanently.

The 210 remaining OOM kills were real and were investigated individually. Nine were genuine leaks, which had been invisible in the noise of 2,800.

Fix 3: remove CPU limits.

This was contested and was rolled out to 20 services first.

                        with CPU limits    without
p99 latency (flagship)      890ms           340ms
CPU throttling               18%             0%
node CPU utilisation         38%             44%
noisy-neighbour incidents     0               2 (in 6 weeks)

p99 latency dropped 62 percent on a service whose mean CPU usage was 0.4 cores against a 2-core limit. It was being throttled because its work was bursty: a request would consume its 200ms quota in 30ms of wall clock and then wait 70ms for the next period.

The two noisy-neighbour incidents were real and were the cost. Both were a single service consuming most of a node's CPU during a batch job, and both were resolved by CPU requests (which give proportional shares under contention) rather than by reinstating limits.

Fix 4: eliminate BestEffort.

apiVersion: v1
kind: LimitRange
metadata: {name: default-requests}
spec:
  limits:
  - type: Container
    defaultRequest: {cpu: "50m", memory: "128Mi"}
    default:        {memory: "128Mi"}      # memory limit only; no CPU limit

A LimitRange with defaultRequest means no pod is ever BestEffort by accident, which matters because BestEffort is evicted first and is what most "why did my pod disappear" tickets turned out to be.

Final:

                              before      after
nodes                         60          44        (-27%, ~$31k/mo)
OOMKilled restarts            2,800/mo    180/mo
evicted pods                  1,200/mo    35/mo
BestEffort pods               41          0
p99 latency (flagship)        890ms       340ms
CPU throttling (fleet mean)   14%         0.3%
cluster allocated (CPU)       94%         52%
node CPU utilisation          31%         44%

Fewer nodes, fewer restarts and better latency simultaneously, which is unusual and is the signature of a system where the numbers had never been measured. Nothing here was a trade-off; it was correcting values that had been copied.

The transferable diagnostic is the gap between allocated and utilised. Ninety-four percent allocated against 31 percent utilised is not a capacity problem, it is a requests problem, and no amount of adding nodes fixes it. Those two numbers next to each other tell you immediately whether to buy hardware or fix manifests.

Production evidence

Kubernetes documentation defines the QoS classes and the eviction ordering, including that within Burstable the ordering is by usage relative to request. The behaviour is specified rather than emergent.

The Vertical Pod Autoscaler exists specifically because humans set these numbers badly. Its recommender component does what the script above does (percentiles over historical usage) and it can apply them automatically. Running VPA in recommendation-only mode is the cheapest possible version of this exercise and it is under-used.

Google's Borg paper (Verma et al., EuroSys 2015) reports that users systematically over-request resources and describes resource reclamation to recover the difference, which is the same finding at a much larger scale and a decade earlier.

The CPU-limits debate is public and unresolved in the community. Tim Hockin (a Kubernetes maintainer) has argued publicly against CPU limits for most workloads, and the counter-argument for predictable capacity planning and multi-tenant isolation is also represented. That a maintainer argues against a feature's common use is worth knowing, because the default advice to set both requests and limits predates the throttling evidence.

The CFS throttling bug (fixed in kernel 5.4, Dave Chiluk's patches) caused throttling at utilisation well below the quota, and many clusters ran affected kernels for years. Knowing this exists explains historical measurements that appear impossible.

Karpenter and Cluster Autoscaler both scale on requests, not usage, which is the mechanism by which bad requests become a direct cloud bill: over-requested pods trigger node provisioning for capacity that will never be used.

The debate

Should you set CPU limits? For most services, no. A CPU limit produces throttling in bursty workloads at average utilisation far below the limit, because the quota is enforced per 100ms period rather than smoothly, and the latency cost is real (890ms to 340ms p99 in the worked example). CPU requests already give proportional shares under contention, which is the isolation people believe limits provide.

The legitimate cases for CPU limits: strict multi-tenancy where you must be able to promise a tenant cannot exceed an entitlement, batch workloads where predictability matters more than latency, and any node with the static CPU manager policy where Guaranteed QoS buys exclusive cores. Outside those, my position is requests without CPU limits, with node autoscaling as the response to genuine saturation.

Should memory requests equal limits? For anything you care about, yes. Memory is incompressible, so the "burst" a higher limit permits is a burst you may not be able to reclaim, and the eviction ordering punishes pods using more than their request. Setting them equal gets Guaranteed QoS and the back of the eviction queue. The cost is packing efficiency, and it is worth it for the class of failure it removes.

Is BestEffort ever right? For genuinely disposable batch work on spare capacity, and almost nowhere else. It is evicted first, its oom_score_adj is 1000, and it usually exists by accident rather than by choice. A LimitRange with defaultRequest is the fix, and it should be in every namespace.

How much should you over-commit? Requests-to-allocatable of 60 to 80 percent is a reasonable working range for a mixed workload, which leaves headroom for bursts and node failure. Above 90 percent you cannot tolerate losing a node without evictions. The number to watch is the gap between allocated and utilised: a large gap means requests are wrong, and a small gap with high allocation means you genuinely need capacity.

Should VPA set these automatically? In recommendation mode, yes, everywhere: it is free and the recommendations are better than most manual values. In auto mode, cautiously, because VPA evicts pods to apply new values, which for a stateful or slow-starting service is disruptive. In-place pod resize (beta in recent versions) removes that objection and makes auto mode much more attractive.

Follow-up Q&A

"What is the difference between a request and a limit?"

A request is what the scheduler uses to decide placement and is a claim on the node's allocatable capacity; a limit is what the kernel enforces at runtime through cgroups. They behave completely differently at the ceiling: exceeding a CPU limit throttles the process until the next 100ms period, and exceeding a memory limit invokes the cgroup OOM killer with exit 137 and no graceful shutdown. CPU is compressible and memory is not, and that asymmetry drives most of the sizing advice.

"How are QoS classes assigned and what do they do?"

Derived, not declared. Guaranteed means requests equal limits for every resource in every container; Burstable means requests are set and lower than limits; BestEffort means neither is set. They determine eviction order under node pressure: BestEffort first, then Burstable ordered by how far each pod exceeds its memory request, then Guaranteed. So a pod requesting 500Mi and using 3Gi is evicted before one requesting 2Gi and using 2.1Gi.

"Should you set CPU limits?"

For most services, no. Requests already give proportional CPU shares under contention, which is the isolation people think limits provide, and limits add throttling that hits bursty workloads at average utilisation far below the limit, because quota is enforced per 100ms period. One service with a 0.4-core mean against a 2-core limit was throttled 18 percent of periods and its p99 was 890ms; removing the limit took it to 340ms. Set them for strict multi-tenancy, batch predictability, or Guaranteed QoS with exclusive cores.

"How do you size these?"

From observation. Memory request at the p99 of container_memory_working_set_bytes over a couple of weeks, plus about 20 percent, with the limit equal to it for Guaranteed QoS. CPU request at the p50 to p90 of the usage rate. Use working set rather than container_memory_usage_bytes, because the latter includes reclaimable page cache and drifts toward the limit, which is the most common cause of memory limits set several times higher than necessary. VPA in recommendation mode does exactly this for free.

"Your cluster is 94 percent allocated and 31 percent utilised. What does that mean?"

Requests are wrong, and it is not a capacity problem. The scheduler is committing against requests, so it thinks the nodes are full while they are idle, and no amount of adding nodes fixes it because the new nodes fill up on paper too. Right-sizing requests from observed usage in one case took allocation from 94 to 52 percent and node count from 60 to 44 while utilisation went up. Those two numbers side by side tell you whether to buy hardware or fix manifests.

"Why do pods get evicted when nothing is obviously wrong?"

Usually a memory request set far below actual usage. Eviction within the Burstable tier is ordered by how far a pod exceeds its memory request, so a pod with a copied 128Mi request using 3Gi is permanently at the front of the queue and gets evicted every time the node comes under any pressure. It presents as random restarts. The fix is a request that reflects real usage, and setting the limit equal to it for Guaranteed QoS.

Common misconceptions

"A request reserves the resource." It is a scheduling claim. At runtime a container can use whatever is free up to its limit, and under contention CPU is shared in proportion to requests rather than allocated.

"Setting both requests and limits is best practice." For memory, setting them equal is right. For CPU, setting a limit introduces throttling at average utilisation well below the limit, and the default advice predates the measurements.

"OOMKilled means a memory leak." Most often it means the limit was copied from another service's manifest. In one audit, 2,800 monthly OOM kills contained nine genuine leaks; the rest were mis-sized limits, and the leaks had been invisible in the noise.

"BestEffort pods are just unconfigured." They are first in the eviction queue with an oom_score_adj of 1000. A namespace LimitRange with defaultRequest prevents them existing by accident.

"Use container_memory_usage_bytes for sizing." It includes reclaimable page cache and trends toward the limit under normal operation. The OOM killer considers the working set, and so should you.

Interview delivery note

Say this verbatim: "Requests are for the scheduler, limits are for the kernel, and CPU and memory behave oppositely at the limit: CPU throttles and memory kills. So I set memory request equal to limit for Guaranteed QoS, and I usually do not set a CPU limit at all, because requests already give proportional shares under contention and limits add throttling at utilisation well below the limit." The distinction, the asymmetry, and a committed position on the contested part.

The senior-versus-staff separator is reading allocated against utilised. A senior engineer sizes resources from metrics. A staff engineer sees 94 percent allocated and 31 percent utilised and says immediately that this is a requests problem rather than a capacity problem, that adding nodes will not help because the new nodes fill on paper too, and that right-sizing took node count down 27 percent while utilisation went up. Two numbers, one diagnosis.

The second signal is the eviction ordering within the Burstable tier. Knowing that pods are evicted by how far they exceed their memory request, not by absolute usage, explains the "random restarts" class of ticket completely: a copied 128Mi request on a service using 3Gi is permanently first in the queue.

The Staff & Team Lead Interview Reference