runc vs gVisor vs Firecracker vs Kata

What it is

Four ways to run a container, differing in where the boundary between the workload and the host kernel sits.

runc                    gVisor                  Kata / Firecracker
┌──────────┐            ┌──────────┐            ┌──────────┐
│ workload │            │ workload │            │ workload │
├──────────┤            ├──────────┤            ├──────────┤
│          │            │  Sentry  │            │  GUEST   │
│          │            │ (userspace│            │  KERNEL  │
│          │            │  kernel) │            ├──────────┤
│          │            ├──────────┤            │   VMM    │
├──────────┤            ├──────────┤            ├──────────┤
│   HOST   │            │   HOST   │            │   HOST   │
│  KERNEL  │            │  KERNEL  │            │  KERNEL  │
└──────────┘            └──────────┘            └──────────┘
 ~350 syscalls           ~70 syscalls            ~40 (KVM ioctls)
 reachable               reachable from          reachable from
                         the workload            the guest

Firecracker is a VMM, not a container runtime. It is a minimal alternative to QEMU (about 50k lines instead of over a million), and Kata Containers is the runtime that can use it. Saying "Firecracker vs Kata" is a category error; the real comparison is Kata-with-QEMU against Kata-with-Firecracker, or Kata against gVisor.

What this is confused with: the idea that these are drop-in security upgrades. gVisor changes syscall performance by up to an order of magnitude for syscall-heavy workloads, and Kata changes the storage and network path. Both are real trades, and choosing one without measuring your workload's syscall profile is how teams end up reverting.

The problem it solves

Containers share the host kernel, so the kernel's syscall interface is the attack surface. Roughly 350 syscalls, many with a long history of privilege-escalation bugs, all reachable from any container.

For your own code that is usually acceptable: an attacker needs an application vulnerability first, and the primitives on the container security page narrow what follows.

For genuinely untrusted code it is not acceptable, and the canonical cases are the same everywhere: running customer-submitted code (CI, serverless functions, notebooks, LLM code interpreters), multi-tenant platforms where tenants must not reach each other, and any regulatory context requiring a stronger boundary than a shared kernel.

The historical record makes the case concretely. Every container escape listed on the security page (runc /proc/self/exe, cgroup release_agent, Dirty Pipe, the runc fd leak) worked through the shared kernel. A guest kernel or a syscall interception layer would have contained all of them, because the exploited interface was not reachable.

Mechanics

runc: the baseline

runc reads an OCI spec and does what the namespaces and cgroups page describes by hand: creates namespaces, sets up the rootfs with pivot_root, applies cgroups, drops capabilities, installs seccomp, and execves the entrypoint. Then it exits: runc is not a supervising process, the container's process is reparented and the shim tracks it.

Every syscall goes straight to the host kernel. That is the performance story and the security story in one sentence.

gVisor: a kernel in userspace

workload
   │  syscall
   ▼
┌────────────────────────────────────────┐
│  Sentry  (Go, runs unprivileged)       │
│    implements ~200 Linux syscalls      │
│    in userspace: file, network, memory │
└──────────────┬─────────────────────────┘
               │ a SMALL set of host syscalls (~70), seccomp-restricted
               ▼
          HOST KERNEL

The Sentry intercepts syscalls (via ptrace historically, now KVM or systrap) and implements them itself. The workload's syscalls are handled by Go code, not by the host kernel.

Two consequences that decide adoption:

Performance depends entirely on syscall frequency. A CPU-bound workload is nearly unaffected because it barely syscalls. An I/O-heavy or network-heavy workload pays 2 to 10x per syscall.

Measured, relative to runc:
  CPU-bound (compression):        0.97x     (essentially free)
  memory allocation heavy:        0.92x
  small-file I/O:                 0.35-0.55x
  network throughput:             0.40-0.70x
  syscall microbenchmark:         0.10-0.30x
  process creation (fork/exec):   0.25x

Compatibility is about 90 percent. Unimplemented syscalls return ENOSYS, so the failures are specific rather than general: some io_uring usage, certain /proc and /sys entries, some ptrace-based tooling, and anything needing raw kernel interfaces. Most applications work; profilers, debuggers and eBPF tooling frequently do not, which is worth knowing because it changes how you operate the workload.

Kata: a real kernel, in a lightweight VM

pod  ->  Kata runtime  ->  VMM (Firecracker / QEMU / Cloud Hypervisor)
                             └─ guest kernel (minimal, ~5 MB)
                                  └─ kata-agent
                                       └─ your container

Each pod gets its own kernel. The host kernel sees a VM, and the only interface is KVM ioctls, which is a much smaller and much better-audited surface than 350 syscalls.

Firecracker is what makes this viable at container density: written for AWS Lambda and Fargate, it strips the device model to a minimum (virtio-net, virtio-block, a serial console, a one-button keyboard controller for reset) and starts a microVM in about 125 ms with roughly 5 MB of overhead.

Firecracker vs QEMU:
  lines of code:        ~50k          vs  >1.4M
  boot time:            ~125 ms       vs  ~500-1500 ms
  memory overhead:      ~5 MB         vs  ~50-130 MB
  device model:         minimal       vs  full PC emulation

Compatibility is complete, because it is a real Linux kernel. The costs are elsewhere: I/O crosses a virtio boundary, memory cannot be shared with the host page cache in the usual way, and some Kubernetes features (host networking, certain volume types, privileged sidecars) do not apply.

Choosing, as a decision procedure

Is the code trusted (yours, or a vetted dependency)?
   YES -> runc, with the security baseline. Do not pay for a sandbox
          you do not need.
   NO  -> continue

Is the workload syscall-heavy (I/O, network, process spawning)?
   YES -> Kata + Firecracker. gVisor's syscall tax lands directly on you.
   NO  -> continue

Do you need full kernel compatibility (eBPF, io_uring, unusual syscalls,
profiling tools)?
   YES -> Kata + Firecracker
   NO  -> gVisor. Lower memory overhead and simpler operationally
          (no nested virtualisation requirement).

Is nested virtualisation available? (bare metal, or a cloud instance type
that supports it)
   NO  -> gVisor is the only option; Kata needs KVM.

That last constraint is the one that decides it in practice. Kata needs KVM, so on most cloud VMs you need bare-metal instances or a provider that exposes nested virtualisation. gVisor runs anywhere.

Mixing runtimes in one cluster

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata: {name: gvisor}
handler: runsc
scheduling:
  nodeSelector: {sandbox: gvisor}      # only nodes that have runsc
---
apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata: {name: kata-fc}
handler: kata-fc
scheduling:
  nodeSelector: {sandbox: kata}

spec:
  runtimeClassName: gvisor            # per-pod choice

RuntimeClass with a node selector is the right shape: sandboxed workloads land on nodes that have the runtime, and trusted workloads stay on ordinary nodes with runc. Running everything sandboxed is paying a tax on code you wrote and trust.

A worked example: a notebook platform, three runtimes measured

A data platform running customer-authored Jupyter notebooks. Arbitrary Python, arbitrary packages, arbitrary code, with access to customer data. About 4,000 notebook sessions a day across 40 nodes.

Starting position: runc with the security baseline (non-root, all capabilities dropped, RuntimeDefault seccomp, read-only root).

A security review flagged the obvious: pip install of an arbitrary package means arbitrary code, the seccomp profile still permits roughly 350 syscalls, and the platform had customer data on the same nodes.

They measured all three, on the actual workload, over two weeks with mirrored traffic:

                          runc      gVisor    Kata+Firecracker
notebook cold start        1.4s      2.1s        2.6s
notebook warm start        0.3s      0.4s        0.4s
pandas read_csv (500MB)    4.2s      11.8s       4.9s
numpy matrix multiply      8.1s      8.3s        8.2s
model training (1 epoch)   94s       112s        97s
pip install (scikit-learn) 18s       71s         22s
network throughput         9.4 Gb/s  3.1 Gb/s    8.1 Gb/s
memory overhead per pod    0 MB      38 MB       11 MB
pods per 64GB node         52        41          48

The pip install number decided it. Package installation is dominated by file creation, stat, open, write and process spawning, which is the exact profile gVisor taxes most. Notebook users install packages constantly, and a 4x regression on the operation they perform most visibly was not acceptable.

The read_csv number is the same story: 2.8x slower under gVisor, because it is small-read-heavy I/O.

And the numpy number is the counter-case: 8.1 to 8.3 seconds, essentially free, because a matrix multiply barely syscalls at all. gVisor's cost is entirely a function of syscall frequency, and reporting a single "gVisor is X percent slower" number is meaningless without saying which workload.

They chose Kata with Firecracker, and hit three operational problems:

Problem 1: nested virtualisation. Their nodes were standard cloud VMs without KVM.

options considered:
  bare-metal instances:      available, ~1.9x the cost per vCPU
  nested-virt instance types: available in some regions only
  -> chose a dedicated bare-metal node pool for notebooks only

cost impact:  notebooks moved to bare metal, +34% on that node pool,
              which was 40 of 210 nodes cluster-wide: +6.5% total compute

Problem 2: image pull and boot amplification. Each pod boots a kernel and mounts the image over virtio.

cold start (uncached image):    2.6s -> measured 14s for large ML images (4-8 GB)

The fix was a combination of pre-pulled images on the node pool, virtio-fs with DAX for page-cache sharing, and a pool of pre-booted microVMs:

warm microVM pool:   20 pre-booted VMs per node, claimed on demand
cold start:          14s -> 1.9s (claim a warm VM, then start the container)

Pre-booting is what makes microVM-per-pod feel like a container, and it is the same technique Lambda uses.

Problem 3: observability tooling stopped working.

lost:      node-level eBPF profiling of notebook processes (they are in a
           different kernel now)
           `kubectl exec` behaved differently for some debugging flows
           host-level `perf` could not see guest processes
gained:    nothing on the observability side
mitigation: an agent inside the guest, shipping to the same backend

This is the under-discussed cost. A guest kernel means host-level tooling cannot see in, and eBPF-based observability is exactly the tooling most platform teams have invested in.

Final:

                              before (runc)   after (Kata+FC)
host kernel syscall surface   ~350            KVM ioctls only
escape from notebook to host  plausible       requires a VM escape
notebook cold start           1.4s            1.9s (warm pool)
pandas/numpy performance      baseline        97-100%
pip install                   18s             22s
pods per node                 52              48
compute cost                  baseline        +6.5% (bare-metal pool)
host-level eBPF profiling     worked          replaced with in-guest agent
security review               blocked         approved

Six and a half percent more compute and 0.5 seconds of cold start, in exchange for a kernel boundary between customer code and customer data. For this workload that was clearly worth it, and the numbers are what made it a decision rather than an argument.

The transferable finding: gVisor's cost is not a single number. The same platform measured 0.97x on numpy and 0.25x on pip install, and a team that benchmarked only the compute-heavy path would have chosen gVisor and shipped a 4x regression on the operation users notice most. Benchmark the syscall-heavy path specifically, because that is where the entire cost lives.

Production evidence

Firecracker powers AWS Lambda and Fargate, and the NSDI 2020 paper documents the design goals: microVM boot in about 125 ms, roughly 5 MB of memory overhead, and thousands of microVMs per host. That is the reference deployment for microVM-per-workload at scale.

gVisor runs Google Cloud Run, App Engine and Cloud Functions, and Google's published material is candid about the performance profile: syscall-heavy workloads pay, compute-heavy workloads do not. Their guidance to benchmark your own workload is the same conclusion as the worked example.

Kata Containers merged Intel Clear Containers and Hyper runV, is an OpenInfra Foundation project, and supports QEMU, Firecracker and Cloud Hypervisor as VMMs. Alibaba, Baidu and others run it in production for multi-tenant isolation.

Fly.io, Modal, E2B and most LLM code-interpreter products run Firecracker microVMs, which is convergent evidence for the specific case of executing untrusted code: when the code is arbitrary and the boundary matters, the industry has settled on a guest kernel.

AWS's own positioning is instructive: Lambda uses Firecracker for tenant isolation, and Fargate moved from a shared-kernel model to Firecracker in 2019. A provider with strong incentives to minimise overhead chose the heavier boundary for multi-tenant code execution.

RuntimeClass is the Kubernetes-native mechanism for per-pod runtime selection and is GA, which means mixing runtimes in one cluster is a supported pattern rather than a workaround.

The debate

Do you need a sandbox at all? For code you wrote and dependencies you vetted, usually not. The security baseline (non-root, no capabilities, RuntimeDefault seccomp, read-only root, user namespace) is a reasonable boundary for semi-trusted workloads, and a sandbox costs performance and operational complexity. The line is whether an attacker needs an application vulnerability first. If they can simply submit code, you need a stronger boundary.

gVisor or Kata? Measure your syscall profile, because that is the whole decision. gVisor is lighter (no nested virtualisation, less memory per pod, simpler operationally) and taxes syscalls by 2 to 10x. Kata is near-native for syscalls and fully compatible, and needs KVM and more memory. My default for untrusted code is Kata with Firecracker, because untrusted code is usually doing I/O and spawning processes, and because full compatibility removes an entire class of "this package does not work" support burden.

Is Firecracker's small codebase a real security argument? Partly. 50k lines against 1.4M is a genuinely smaller audit surface, and it is written in Rust, and the boundary that matters is KVM rather than the VMM. Firecracker's own threat model is explicit that it depends on KVM's correctness. The argument is real and it is about the VMM's surface, not about the virtualisation boundary itself.

What about the observability cost? It is the cost teams do not price in. A guest kernel means host-level eBPF cannot see guest processes, perf does not work across the boundary, and the profiling investment most platform teams have made stops applying to sandboxed workloads. Budget for in-guest agents, and note that gVisor has the same problem differently: the Sentry is what runs, so host tooling sees Go goroutines rather than your application.

Should you sandbox everything? No. Running your own trusted services under gVisor pays a syscall tax for a threat that requires an application vulnerability you can address directly. RuntimeClass with a node selector is the right shape: sandboxed node pools for untrusted workloads, ordinary nodes with runc for everything else.

Is the performance gap closing? gVisor's systrap platform substantially improved on ptrace and KVM modes, and Firecracker and Cloud Hypervisor keep reducing boot time and overhead. The structural costs remain: gVisor intercepts syscalls in userspace and Kata crosses a virtio boundary for I/O. Re-benchmark on your workload rather than trusting a figure from two years ago, and expect the shape of the trade to persist even as the magnitudes shrink.

Follow-up Q&A

"What is the difference between these four?"

Where the boundary sits. runc uses namespaces and cgroups, so the workload's syscalls go straight to the host kernel: roughly 350 syscalls of attack surface. gVisor puts a userspace kernel (the Sentry) in between, implementing most syscalls itself and making only about 70 host calls. Kata gives each pod a real guest kernel in a lightweight VM, so the host sees only KVM ioctls. Firecracker is not a runtime at all: it is a minimal VMM that Kata can use instead of QEMU, at about 50k lines and 125 ms boot.

"When would you use gVisor over Kata?"

When nested virtualisation is unavailable, which is the common case on standard cloud VMs, since Kata needs KVM. Also when memory overhead per pod matters at density (roughly 38 MB versus 11 MB in one measurement, though the ordering depends on configuration) and when the workload is compute-bound rather than syscall-bound. The disqualifier is a syscall-heavy workload: gVisor taxes syscalls 2 to 10x, so file I/O, networking and process spawning pay heavily.

"What is gVisor's actual performance cost?"

Entirely a function of syscall frequency, so a single number is meaningless. On one workload, numpy matrix multiply was 0.97x (essentially free, because it barely syscalls) and pip install was 0.25x (4x slower, because package installation is dominated by file creation and process spawning). A team that benchmarked only the compute path would have chosen gVisor and shipped a 4x regression on the operation users notice most. Benchmark the syscall-heavy path specifically.

"Why does Firecracker matter?"

It makes VM-per-workload viable at container density. About 50k lines against QEMU's 1.4 million, a minimal device model, roughly 125 ms boot and 5 MB of overhead, written in Rust. It powers Lambda and Fargate, which is a provider with strong incentives to minimise overhead choosing a guest kernel for multi-tenant code execution.

"Do you need any of this?"

For code you wrote and dependencies you vetted, usually not: the security baseline is a reasonable boundary when an attacker needs an application vulnerability first. The line is whether they can simply submit code. Running customer notebooks, CI jobs, serverless functions or LLM-generated code means arbitrary execution by design, and there the shared kernel is the wrong boundary.

"What do you lose operationally?"

Observability, mostly, and it is the cost people do not price in. With Kata, host-level eBPF cannot see guest processes and perf does not cross the boundary, so the profiling investment most platform teams have made stops applying and you need in-guest agents. gVisor has the same problem differently: host tooling sees the Sentry's goroutines rather than your application. Also image pull and boot amplification with Kata, which pre-booted microVM pools address, and that is the same technique Lambda uses.

What is OverlayFS, and what does it explain about container behaviour? OverlayFS is the union filesystem the Linux kernel provides and the storage driver essentially every container runtime now uses (Docker's overlay2, containerd, CRI-O). It presents a single merged filesystem view assembled from a stack of read-only lower directories, one writable upper directory, and a merged mount point that the container sees. Each image layer is a lower directory; the container's writable layer is the upper one.

Three container behaviours fall directly out of that design, and being able to derive them rather than memorise them is the signal. Layer caching works because lower directories are immutable and content-addressed, so two images sharing a base share the same on-disk directories and pull only what differs. Copy-on-write means the first write to a file that exists in a lower layer copies the whole file up to the upper directory before modifying it, so writing one byte to a 2 GB file costs a 2 GB copy and a latency spike; that is why database data directories belong on a volume rather than in the container's writable layer, and it is the concrete answer to "why not just run Postgres in the container filesystem". And deletions are whiteouts: removing a file that lives in a lower layer creates a special marker in the upper layer rather than freeing anything, which is why RUN rm -rf /secrets in a later Dockerfile line does not remove the secret from the image. The file is still in the earlier layer and anyone with the image can read it, which is the single most common way credentials leak through container images, and the fix is a multi-stage build or never adding the file, not deleting it later.

Common misconceptions

"Firecracker is a container runtime." It is a VMM, an alternative to QEMU. Kata is the runtime that can use it. The real comparisons are Kata-with-Firecracker against Kata-with-QEMU, or Kata against gVisor.

"gVisor is slower by X percent." Its cost is proportional to syscall frequency and ranges from about 3 percent on compute-bound work to 4x on package installation. Any single figure is describing one workload.

"A sandbox replaces the security baseline." Non-root, dropped capabilities, seccomp and read-only root still apply inside a sandbox. They are layers, and a sandbox is the outermost one.

"Sandboxing everything is safer." It pays a performance and operational tax on trusted code for a threat that requires an application vulnerability. RuntimeClass exists so you can sandbox selectively.

"Kata is just a VM, so it is slow." With Firecracker, boot is about 125 ms and syscall performance is near-native because it is a real kernel. The costs are I/O crossing virtio and memory overhead, not syscall latency, and a pre-booted pool removes most of the start-up cost.

Interview delivery note

Say this verbatim: "The question is where the boundary sits. runc leaves 350 host syscalls reachable, gVisor puts a userspace kernel in front so about 70 are, and Kata gives each pod a real guest kernel so the host sees only KVM ioctls. For untrusted code I default to Kata with Firecracker, because untrusted code tends to be syscall-heavy and gVisor's tax is entirely proportional to syscall frequency." The architecture, the surface each exposes, and a committed default with its reason.

The senior-versus-staff separator is knowing that gVisor's cost has no single number. A senior engineer describes the architectures correctly. A staff engineer says the cost is proportional to syscall frequency, gives the spread (0.97x on numpy, 0.25x on pip install), and points out that benchmarking only the compute path leads a team to ship a 4x regression on the operation users see most. Insisting on benchmarking the syscall-heavy path is the practical judgement.

The second signal is pricing the observability loss. Saying "a guest kernel means host-level eBPF cannot see guest processes, so the profiling investment stops applying and you budget for in-guest agents" shows you have operated one of these rather than evaluated it, and it is the cost that gets discovered after the migration.

Further reading

• Agache et al., "Firecracker: Lightweight Virtualization for Serverless Applications" (NSDI 2020), for the design goals and the Lambda deployment.
• gVisor's documentation on its architecture and platforms (ptrace, KVM, systrap), and its published performance guidance.
• Kata Containers documentation on VMM selection and the virtio-fs / DAX configuration for image sharing.
• Kubernetes RuntimeClass documentation, for per-pod runtime selection with node scheduling.