Serverless or containers: walk the math

"Serverless or containers for this workload? Walk your math."

What it is

A placement decision across four options that differ in who manages what and how you are billed:

The decision is commonly framed as a philosophy ("we're serverless-first") and it is an arithmetic problem with a threshold. You pay per unit of work with functions and per unit of time with servers, so the crossover is set by utilisation. Below the threshold, paying only for work is cheaper; above it, paying for time is cheaper because the time is fully used.

Commonly confused with "serverless means no servers". There are servers; you do not operate them. And confused with a scaling question: all four scale, they differ in how fast, how granularly, and what you pay while idle.

The problem it solves

Two failure modes motivate having a framework rather than a preference.

Over-provisioned always-on infrastructure for spiky work. A batch job that runs for eight minutes a day on an instance billed for 24 hours is paying roughly 180 times what the work costs.

Functions for steady high-volume work. A service at constant load on a per-invocation model pays a premium on every request forever, plus cold-start latency, plus the architectural constraints (execution time limits, connection management, no local state) that a plain server would not impose.

Both are common, both are expensive, and both come from picking a platform by conviction rather than by utilisation.

Mechanics

The crossover arithmetic

The pricing shapes, with illustrative figures (check current rates; the shape is what matters and it is stable):

Functions:   ~$0.20 per million requests
           + ~$0.0000167 per GB-second of execution

Serverless containers: ~$0.04 per vCPU-hour + ~$0.004 per GB-hour

VMs / nodes: ~$0.04 per vCPU-hour equivalent, plus you pay while idle

Now a concrete comparison. A service at 50 requests per second, 200 ms per request, 512 MB of memory.

Monthly requests
  50 x 86,400 x 30 = 129.6 million

FUNCTIONS
  Request charge:  129.6M x $0.20/M                    = $25.92
  Compute:         129.6M x 0.2 s x 0.5 GB             = 12.96M GB-s
                   12.96M x $0.0000167                 = $216.43
  Total                                                 ~$242/month

CONTAINERS (sized from Little's Law, not from guessing)
  Concurrency = throughput x latency = 50 x 0.2 = 10 in flight
  At ~50 requests/sec/vCPU for this workload -> 1 vCPU sustains it,
  so 2 vCPU + 4 GB for headroom and redundancy across two tasks.
  2 vCPU x 730 h x $0.04                              = $58.40
  4 GB   x 730 h x $0.004                             = $11.68
  Total                                                 ~$70/month

Containers win by roughly 3.5x at this load, and the reason is visible in the numbers: the service is busy most of the time, so paying for time is efficient.

Now change one variable. Same service at 2 requests per second:

FUNCTIONS   5.2M requests -> ~$1 + ~$8.70 compute      = ~$10/month
CONTAINERS  Still need a task running (and two for HA) = ~$70/month

Functions win by 7x. Nothing about the code changed; only the utilisation did.

The crossover, stated generally

Set the two costs equal and solve. With the figures above, the crossover for this memory size lands at roughly 8 to 12 requests per second, which corresponds to roughly 30 to 40 percent utilisation of the equivalent container.

Below ~35 percent average utilisation, functions win. Above it, always-on wins. That single sentence is the answer to the drill, and the reason it holds across providers is that it is a consequence of the billing model rather than of any specific price.

Two adjustments that move the line:

• Committed-use discounts (reserved instances, savings plans) cut always-on cost by 30 to 60 percent, pushing the crossover down to perhaps 20 percent utilisation. If the workload is steady enough to commit, that is a large thumb on the scale.
• Very spiky traffic moves it up, because always-on must be provisioned for peak while functions are billed at average. A workload with a 20x peak-to-trough ratio pays for peak capacity all day.

Cold starts, and when they actually matter

The mitigations: smaller deployment packages, lazy imports so initialisation does not load what a given path does not need, provisioned concurrency (which is paying for always-on, so it moves you toward the container answer anyway), and snapshot restore for JVM runtimes.

When it matters: user-facing synchronous requests at the p99. When it does not: asynchronous processing, scheduled jobs, event handlers where a second is invisible.

The constraints that decide it regardless of cost

Cost is one axis. These are the ones that override it:

Connection management. A function per invocation cannot hold a database connection pool, so 500 concurrent functions become 500 connections and exhaust the database. The fix is a connection proxy, or a data API, and that is a real architectural cost. This is the single most common way a function-based design fails at scale.

Execution time limits. Functions cap out (commonly 15 minutes). Anything longer needs a different platform or decomposition into a state machine.

Local state and warm caches. A function has no reliable in-process cache, so work a server would do once per instance gets done per invocation.

Consistent latency. Cold starts make the tail unpredictable in a way an always-on service is not.

GPU and specialised hardware. Available on VMs and Kubernetes, not on general-purpose function platforms.

The decision framework

Is it event-driven, spiky, and short (< 15 min)?
  and is average utilisation below ~35%?
  and can it tolerate cold-start tail latency?
     -> FUNCTIONS

Is it a long-running service with variable load,
  and do you not want to operate a cluster?
     -> SERVERLESS CONTAINERS

Do you need scheduling control, multi-tenancy, daemonsets,
  service mesh, or portability across clouds?
     -> MANAGED KUBERNETES

Do you need GPUs, specialised hardware, or extreme cost
  optimisation at steady high scale?
     -> VMs WITH COMMITTED-USE DISCOUNTS

And the answer worth volunteering: most real systems are a mixture. The synchronous API on containers, the event handlers and scheduled jobs on functions, the training workload on GPU instances. Presenting it as a single platform choice is the mistake.

A worked example

Three workloads at one company, and the same framework produces three answers.

1. Image thumbnail generation on upload. 40,000 uploads/day, bursty (60 percent arrive in a 3-hour window), 1.5 s per image, 1 GB memory.

Average utilisation of an equivalent always-on task: ~2%
Functions: 40k x 30 = 1.2M invocations/month
           1.2M x 1.5 s x 1 GB = 1.8M GB-s -> ~$30 + $0.24 = ~$30/month
Containers: must be provisioned for the burst -> ~$140/month, idle 98% of the time

Functions, comfortably. Bursty, short, event-driven, latency-tolerant.

2. The main product API. 800 requests/sec sustained, 80 ms p50, 512 MB.

Utilisation of a right-sized fleet: ~65%
Functions: 2.07B requests/month
           $414 request charge + 2.07B x 0.08 x 0.5 x $0.0000167 = $1,383
           Total ~$1,800/month, plus cold starts in the user path, plus
           2,000+ database connections to manage.
Containers: concurrency = 800 x 0.08 = 64 in flight; ~16 vCPU with headroom
            ~$470/month, or ~$250 with a committed-use discount.

Containers, by 4 to 7x, and the connection-count problem would have forced it anyway.

3. Nightly reconciliation batch. Runs 40 minutes, once a day, 8 GB memory.

Functions: exceeds the 15-minute execution limit. Excluded on constraints,
           not on cost.
Serverless containers: 40 min x 30 days = 20 h/month x (4 vCPU + 8 GB)
                       ~$7/month, scales to zero between runs
Always-on VM: ~$180/month for 20 hours of work

Serverless containers, chosen on the execution limit rather than the price, and cheap as a bonus.

The observation to make out loud: the same organisation is correctly using three platforms, and a "serverless-first" or "Kubernetes-everything" policy would have got two of the three wrong. The framework, not the conviction, is the answer.

Production evidence

AWS's own guidance frames Lambda for event-driven and spiky workloads and Fargate or ECS/EKS for long-running services, and the existence of provisioned concurrency is itself an admission that steady load on a per-invocation model wants always-on capacity.

RDS Proxy and equivalent connection-pooling services exist specifically because the connection-per-invocation problem is the most common way function architectures fail against relational databases, which is good evidence for treating it as a first-order constraint rather than a detail.

Lambda SnapStart (snapshot-and-restore for JVM runtimes) exists because multi-second JVM cold starts made functions unusable for a large class of enterprise workloads, and it brought them into the hundreds of milliseconds.

Amazon's Prime Video write-up (2023) is the widely-cited case of a team moving a data-intensive pipeline from distributed serverless components to a single always-on process and reporting a cost reduction of over 90 percent, because orchestration and inter-component data transfer dominated the actual work. It is a data point about fine-grained serverless for high-throughput data processing specifically, not about serverless generally, and quoting it as the latter is a mistake an interviewer may be testing for.

The debate

The case for serverless-first: operational simplicity is worth real money. No patching, no capacity planning, no cluster upgrades, scaling for free. For a small team the engineering time saved can exceed the compute premium by a wide margin, and "we spend nothing operating it" is a legitimate answer even when the compute bill is higher.

The case for containers-first: predictable cost and latency, no execution limits, normal connection pooling, ordinary local caching, and no per-request premium. And platform lock-in is materially lower, because a container runs anywhere.

My position: decide per workload with the utilisation arithmetic, and expect the answer to be a mixture. Below roughly 35 percent average utilisation, functions win on cost; above it, always-on does, and committed-use discounts push the line down further. But do the constraint check first, because execution limits, connection management and cold-start tolerance override cost, and a decision that is right on price and wrong on connections will fail at scale rather than merely cost more.

Functions are the wrong default for a steady high-volume synchronous API, for anything needing a connection pool without a proxy, for long-running work, and for latency-critical paths where cold starts land in the p99. Always-on is the wrong default for genuinely spiky event-driven work, where you are buying idle capacity.

Follow-up Q&A

"Serverless or containers for this workload? Walk your math." Compute both. Functions are per-request plus GB-seconds; containers are vCPU-hours plus GB-hours, and you size the container from Little's Law: concurrency equals throughput times latency. Then compare. The crossover lands around 30 to 40 percent average utilisation, because below that you are buying idle time and above it you are paying a per-request premium on fully-used capacity. Committed-use discounts push the crossover down to roughly 20 percent. Then check the constraints, which can override the cost answer entirely.

"Which constraints override cost?" Connection management, first: a function per invocation cannot hold a pool, so hundreds of concurrent functions exhaust a relational database, and the fix is a proxy or a data API, which is a real architectural cost. Then execution time limits, typically 15 minutes. Then cold-start tolerance in a user-facing path. Then local state and warm caches, which functions cannot rely on. And GPUs or specialised hardware, which general-purpose function platforms do not offer.

"How bad are cold starts really?" It depends on the runtime and it is knowable: 100 to 400 ms for a small interpreted package, 1 to 6 seconds for an uninitialised JVM, and roughly 200 ms for a JVM with snapshot restore. They matter at the p99 of a synchronous user request and are invisible in asynchronous processing. Provisioned concurrency removes them, and it also removes the pricing advantage that made you choose functions, so if you find yourself provisioning a lot of concurrency that is a signal the arithmetic has moved.

"Your service is at 800 requests per second on functions and the bill is huge. What's the first move?" Check utilisation, which at that rate is almost certainly well above the crossover, so moving to always-on containers is likely a 4 to 7x saving. But check the connection count first, because at that concurrency you are probably already running a connection proxy, and the migration removes that too. I would also check whether memory is over-allocated, since the GB-second charge is linear in configured memory and teams routinely over-provision it because memory also controls CPU allocation.

"When is the answer 'a mixture'?" Almost always, and I would say so unprompted. The synchronous API on containers, the event handlers and scheduled jobs on functions, the GPU work on instances. A single-platform policy gets some workloads wrong by construction, and the cost of running two platforms is much lower than people assume once the deployment pipeline handles both.

Common misconceptions

The most common is that serverless is cheaper. It is cheaper at low utilisation and markedly more expensive at high utilisation, and which side you are on is arithmetic rather than opinion.

The second is that the choice is about scaling. All four options scale; they differ in granularity, speed, and what you pay while idle.

The third is that the cost comparison is the decision. Connection management and execution limits override it, and a design that is right on price and wrong on connections fails at scale rather than merely costing more.

Interview delivery note

Do the arithmetic out loud, because that is the drill: "Functions are per-request plus GB-seconds; containers are vCPU-hours, and I'd size the container from Little's Law, so concurrency is throughput times latency. At 50 requests a second and 200 milliseconds that's 10 in flight, roughly two vCPU with headroom, about $70 a month. The same traffic on functions is about $240. But at 2 requests a second the container still costs $70 and the functions cost $10."

Then state the general rule: "So the crossover is around 30 to 40 percent utilisation, and committed-use discounts push it down toward 20. Below that, pay per unit of work; above it, pay per unit of time."

The depth signal is checking constraints before cost: "before any of that I'd check connection management, because a function per invocation can't hold a pool and hundreds of concurrent functions will exhaust the database. That's the most common way this decision fails, and it fails at scale rather than showing up on the bill." And close with the mixture, because a single-platform answer is the weaker one.

Further reading

• AWS Lambda and Fargate pricing documentation, plus the provisioned concurrency and SnapStart pages, for the mechanics behind the arithmetic.
• RDS Proxy documentation, for why connection management is a first-order constraint rather than a detail.
• Amazon Prime Video Tech Blog, "Scaling up the Prime Video audio/video monitoring service and reducing costs by 90%" (2023), read in full rather than by headline.
• AWS Well-Architected Framework, cost optimisation pillar, for committed-use discounts and right-sizing as a discipline.