Multi-cloud, egress, and data gravity

What it is

Multi-cloud is running production workloads on more than one provider. Portability is the ability to move a workload without rewriting it. They are different, and conflating them is how organisations pay for one while wanting the other.

PORTABILITY LEVELS, easiest to hardest:

  workload      containers, and a Kubernetes API. Genuinely
                portable, and it is the layer everyone points
                at.
  build/deploy  CI, IaC, images. Portable with effort; the
                IaC is provider-specific even in a common
                language.
  data          hard. Terabytes have a transfer cost and a
                transfer time, and the managed database you
                used is not the one over there.
  control plane very hard. IAM, networking, quotas,
                observability, secrets, and every managed
                service you built on.

Data gravity is the observation that data attracts compute: once a petabyte lives somewhere, moving it costs money and weeks, so the next system gets built next to it. Egress pricing is the mechanism that makes gravity a financial force rather than a physical metaphor.

What this is confused with: egress as the internet bill. The surprising charges are almost never internet egress. They are cross-availability-zone traffic, NAT gateway data processing, and inter-region replication, all of which are internal and none of which anyone budgeted for.

Also confused: multi-cloud and a disaster recovery strategy. A second provider you cannot fail over to in an emergency is not a DR plan; it is a second estate.

The problem it solves

Multi-cloud solves specific, nameable problems and does not solve lock-in as an abstract concern.

REASONS THAT ARE REAL
  a regulator or a customer contract requires a specific
    provider for a specific workload
  an acquisition brought an estate on another provider
  one provider has a service with no equivalent (a specific
    accelerator, a specific managed model, a sovereign region)
  genuine negotiating leverage at a spend where the discount
    is worth the cost
  a customer sells into a market where your provider is
    unavailable

REASONS THAT ARE NOT
  "avoiding lock-in" with no named risk
  resilience against a provider-wide outage, which is rarer
    than the outages your own multi-cloud complexity will
    cause
  a belief that portability is free if you use Kubernetes

And the cost of multi-cloud is continuous, not one-off:

- LOWEST COMMON DENOMINATOR. To stay portable you avoid the
  managed services that make a cloud worth using, so you
  operate your own database, queue and cache. That is the
  largest cost and it is paid every day.
- TWO OF EVERYTHING. Two IAM models, two networking models,
  two quota systems, two support relationships, two sets of
  expertise, two on-call knowledge bases.
- EGRESS BETWEEN THEM, which is a per-gigabyte tax on every
  cross-provider interaction.
- HIRING. An engineer deeply fluent in both is rare and
  expensive; two half-fluent teams is the usual reality.

The egress problem is separate and much more commonly the actual finding:

A monthly bill with a $47,000 line item nobody recognises.

  internet egress               $6,200
  cross-AZ data transfer       $22,400
  NAT gateway processing       $14,100
  inter-region replication      $4,300

Three of the four are internal traffic. None of them appear in
any architecture diagram. All of them are consequences of
placement decisions nobody costed.

Mechanics

The honest multi-cloud position

DEFAULT: one primary provider, used properly, with managed
services, plus a documented and rehearsed exit.

WHAT AN EXIT STRATEGY ACTUALLY CONTAINS:
  - data export in an open format, tested. Not "we could
    export", but a scheduled export of the real dataset with a
    measured duration.
  - workloads in containers with no provider-specific runtime
    assumptions
  - IaC that describes the intent, accepting that the modules
    are provider-specific
  - a written inventory of every managed service used, with
    the equivalent or the replacement plan for each
  - a measured estimate of the migration cost and duration

That inventory is the deliverable. It converts "we are locked
in" from an anxiety into a number, which is what makes the
conversation tractable.

PAY FOR PORTABILITY ONLY WHERE A NAMED REQUIREMENT EXISTS.

Abstraction layers do not give you portability:

Terraform gives you one LANGUAGE, not one SEMANTIC. An AWS
module and a GCP module for "a Postgres database" are not
interchangeable: different parameters, different networking,
different backup semantics, different failover behaviour.

Kubernetes gives you a portable workload API and does not
cover storage classes, load balancer behaviour, IAM
integration, or any managed service you call.

Crossplane and similar tools give you a consistent control
plane for provisioning and still leave the semantics
different.

The realistic claim: these reduce the migration from a rewrite
to a large project. That is worth something and it is not
portability.

The egress cost model

The four categories, roughly ordered by how often they
surprise people:

CROSS-AVAILABILITY-ZONE
  charged per GB, and on some providers in BOTH directions.
  Order of magnitude: ~$0.01/GB each way.
  Why it surprises: a service mesh or a Kubernetes service
  distributes traffic across zones by default, so roughly
  two thirds of your internal traffic crosses a zone
  boundary in a three-zone cluster. Nobody chose that.

NAT GATEWAY DATA PROCESSING
  a per-GB charge on everything a private subnet sends to the
  internet, ON TOP of the egress itself, plus an hourly
  charge per gateway.
  Order of magnitude: ~$0.045/GB processed.
  Why it surprises: pulling container images, calling
  object storage without a VPC endpoint, and third-party API
  traffic all route through it. A workload pulling a 2 GB
  image on every pod start, 500 times a day, is 1 TB/day
  through the NAT.

INTER-REGION
  ~$0.02/GB, and it is the cost of cross-region replication,
  multi-region backups, and a global database's write
  propagation.

INTERNET EGRESS
  ~$0.05-0.09/GB in the first tier, declining with volume,
  and free or heavily discounted to a CDN on most providers.
  Usually the smallest of the four for an internal-heavy
  system, and the one everyone thinks of first.

A recent and material change: following the EU Data Act, the
major providers now waive egress fees for customers
transferring their data OUT when leaving. That removes the
exit-cost argument for lock-in and does nothing about the
day-to-day internal charges above, which is where the money
actually is.

The fixes, in order of size

1. VPC / PRIVATE ENDPOINTS for object storage and other
   provider services.
   Traffic to S3 or equivalent through a gateway endpoint is
   free, and through a NAT gateway is $0.045/GB plus egress.
   This is frequently the single largest line item and the
   cheapest fix: it is a routing table entry.

2. IMAGE CACHING / PULL-THROUGH CACHE in-region and in-VPC,
   so image pulls do not traverse the NAT at all.

3. TOPOLOGY-AWARE ROUTING. Prefer a same-zone endpoint when
   one is healthy, falling back across zones when it is not.
   Kubernetes supports this natively (traffic distribution
   preferences, and previously topology-aware hints), and
   service meshes support locality-weighted load balancing.
   Effect: a chatty service pair goes from ~2/3 of traffic
   crossing a zone to a small fraction.
   THE CAVEAT that must be stated: same-zone preference
   reduces the effective pool for load balancing and can
   create hot spots, and it must fall back cleanly when a
   zone's endpoints are unhealthy or you have converted a
   cost optimisation into an availability problem.

4. COLOCATE CHATTY PAIRS. If two services exchange 100 TB a
   month, the question is why they are two services and, if
   they must be, whether they can share a zone-aware
   placement.

5. COMPRESS AND BATCH. A chatty protocol at 100 TB/month
   compressed 4:1 is 25 TB/month. Unglamorous and it works.

6. CDN for internet egress, which is usually free or
   discounted from origin to CDN and cheaper from CDN to
   user.

Fix 1 is the highest-return and it is a routing table entry, which is why it is worth checking first in any cost investigation.

Data gravity, made concrete

1 PB in object storage.

Moving it to another provider:
  egress at $0.05/GB           = ~$50,000
  transfer at a sustained
    10 Gbps                    = ~9.5 days
  plus the reads, the writes at the destination, and the
  validation

So the data does not move. The next analytics system is built
where the data is. Then the next one. Then the compute that
serves it, because moving compute to data is cheap and the
reverse is not.

That is data gravity, and it is why the practical lock-in is
almost never the compute API. It is the storage location and
the accumulated systems that read from it.

The mitigations, none of which are free:
  - open table formats (Iceberg, Delta) on object storage, so
    the QUERY ENGINE is replaceable even if the bytes are not
  - a documented, tested export path
  - keeping the primary copy in a format any provider can read
  - accepting the gravity and deciding deliberately

A worked example: a $47,000 line item and a multi-cloud proposal

A company at roughly $290,000/month, with a board-level question about provider concentration risk and a proposal to go multi-cloud, plus a data-transfer line item nobody could explain.

The data transfer breakdown, obtained by tagging and flow logs:

  cross-AZ                    $22,400/month
  NAT gateway processing      $14,100
  internet egress              $6,200
  inter-region replication     $4,300
  ---------------------------------------
                              $47,000/month  (16% of the bill)

Cause analysis, which found three placement decisions nobody had made deliberately:

1. NAT GATEWAY, $14,100.
   Of the traffic processed:
     61%  object storage reads and writes from analytics
          jobs, routed through the NAT because no gateway
          endpoint existed
     22%  container image pulls
     17%  genuine third-party API calls

   The 61% was traffic to a service in the same region, going
   out to the internet and back in, and being charged twice
   for the privilege.

   FIX: a gateway VPC endpoint for object storage. One
   Terraform resource, one route table association.
   -$8,600/month, applied in an afternoon.

   Image pulls: a pull-through cache in-region.
   -$3,100/month.

2. CROSS-AZ, $22,400.
   Flow log analysis of the top talking pairs:
     the API gateway to the pricing service: 140 TB/month
     the pricing service to Redis: 95 TB/month
     everything else: the remainder

   Both pairs were load-balanced across three zones with no
   locality preference, so roughly two thirds of every
   conversation crossed a zone boundary.

   FIX: topology-aware routing enabled for both, with health
   fallback across zones.
     cross-zone share of that traffic: ~66% -> ~11%
     -$13,900/month

   THE CAVEAT, tested before rollout: with same-zone
   preference, losing one zone's pricing pods must fail over
   cleanly. Verified in a game day by cordoning a zone; the
   fallback worked and added 0.4ms p99.

3. INTER-REGION, $4,300.
   Cross-region backup replication of everything, including
   16 TB/month of derived data that was regenerable from
   source in under two hours.

   FIX: replicate the source data and the irreplaceable
   artifacts; regenerate the derived data in the DR region on
   demand.
   -$2,900/month.

TOTAL: $47,000 -> $18,500/month. Two weeks of work.

Sixty-one percent of NAT gateway traffic was same-region object storage access routed out to the internet and back, which is the single most common finding in this category and is fixed by a routing table entry.

The multi-cloud proposal, evaluated:

The board question was concentration risk. The proposal was
to run the platform on two providers.

Costed:
  duplicate infrastructure                    +$180k/month
  replacing 9 managed services with
    self-hosted equivalents (database,
    queue, cache, search, secrets, ...)       ~6 engineer-
                                              quarters, then
                                              ~2 engineers of
                                              permanent
                                              operational load
  cross-provider egress for a split
    workload                                  +$30-70k/month
                                              depending on the
                                              split
  two IAM models, two networking models,
    two on-call knowledge bases               unquantified and
                                              large

Against the stated risk: a provider-wide outage.
Historical evidence: the provider's worst regional events in
the preceding three years were regional, not global, and the
company's existing multi-region posture already addressed
those.

REJECTED, and the counter-proposal accepted:

  1. An EXIT INVENTORY: every managed service used, its
     equivalent on two other providers, and a measured
     estimate of the migration effort. 3 weeks of work,
     produced a 14-page document and a number: roughly 3
     engineer-years and 5 to 8 months of elapsed time to
     move the platform, with the data transfer now free
     under the EU Data Act provisions the provider had
     adopted.
  2. A TESTED DATA EXPORT: a quarterly job exporting the
     primary datasets to an open format, with the duration
     measured. First run: 31 hours for 340 TB.
  3. OPEN TABLE FORMAT for the analytics estate (Iceberg on
     object storage), so the query engine is replaceable
     even though the bytes are expensive to move.
  4. A SECOND PROVIDER for exactly one thing: a specific
     managed model service with no equivalent, used through
     an abstraction the team already had.

Cost of the counter-proposal: ~1.5 engineer-quarters and
about $2,000/month.

"Roughly three engineer-years and five to eight months" converted lock-in from an anxiety into a number, and that number was what the board actually wanted: not an escape, an understanding of the exposure.

Two things worth recording:

1. THE EXIT INVENTORY HAD A SECOND USE nobody predicted. It
   became the input to a provider negotiation, because a
   credible, costed exit is the only real leverage in a cloud
   contract. The subsequent renewal improved the discount by
   a low double-digit percentage, which is more than the
   multi-cloud proposal would have saved in negotiating
   leverage and cost roughly nothing.

2. TOPOLOGY-AWARE ROUTING WAS ROLLED BACK ONCE.
   The first rollout enabled it for a service with only two
   replicas per zone. During a deploy, one zone briefly had
   zero healthy endpoints, and same-zone preference sent that
   zone's traffic nowhere for about 20 seconds before
   fallback engaged.
   Fix: a minimum-replica precondition (at least three
   healthy endpoints per zone) before enabling locality
   preference for a service, and the fallback threshold
   tuned.
   The general lesson: a cost optimisation that changes
   routing is a change to the availability posture, and it
   needs the same care as any other routing change.

Production evidence

Cloud egress pricing structures are published by all three major providers, with the consistent shape that internet egress is tiered and declining, inter-region is materially cheaper, and cross-availability- zone transfer is charged per gigabyte, in some cases in both directions.

The EU Data Act, applicable from January 2024 with switching provisions phasing in, required cloud providers to remove charges for switching; AWS, Google Cloud and Microsoft Azure subsequently announced free data transfer out for customers leaving. That removes the exit-egress argument specifically and does not affect ordinary operational transfer charges.

Gateway VPC endpoints for object storage are free of data processing charges on AWS, in contrast to NAT gateway data processing, which is the documented basis for the single largest fix described above.

Kubernetes topology-aware routing (traffic distribution preferences, previously topology-aware hints) and service-mesh locality-weighted load balancing are documented features whose explicit purpose is reducing cross-zone traffic, with documented caveats about endpoint distribution and the need for fallback when a zone's endpoints are insufficient.

Data gravity was named by Dave McCrory and describes the observed tendency of applications and services to accumulate around large datasets; the mechanism is that transfer cost and duration scale with volume while compute placement is nearly free to change.

Open table formats (Apache Iceberg, Delta Lake) decouple the storage layer from the query engine, and their multi-engine support (Spark, Trino, Flink, Snowflake, BigQuery, Athena) is the practical mechanism by which an analytics estate becomes engine-portable without the bytes moving.

Kubernetes as a portability layer covers the workload API and explicitly not storage classes, load balancer semantics, IAM integration or managed services, which is documented in every provider's managed Kubernetes offering as the set of provider-specific integrations.

The debate

Is multi-cloud worth it? For a named requirement, yes: a regulator, a contract, an acquisition, a service with no equivalent. For lock-in anxiety with no named risk, no, because the cost is continuous and the largest part of it is being unable to use the managed services that make a cloud worth paying for. The productive substitute is a costed exit inventory, which converts the anxiety into a number and, in one case, turned out to be better negotiating leverage than the multi-cloud plan itself.

Does Kubernetes make you portable? It makes the workload portable and leaves storage classes, load balancer behaviour, IAM integration and every managed service you call provider-specific. The honest claim is that it reduces a migration from a rewrite to a large project, which is worth something and is not the same as portability.

Is a second provider a DR strategy? Only if you have rehearsed failing over to it, which almost nobody has. A second estate you cannot fail over to in an emergency is two estates. Multi-region on one provider addresses the failures that actually occur, and provider-wide outages are rarer than the outages a cross-provider architecture introduces.

Should you optimise egress or ignore it? Optimise, because in a typical internal-heavy system it is a double-digit percentage of the bill and the largest fixes are configuration rather than architecture. A gateway endpoint for object storage is a routing table entry and was worth $8,600 a month in one case, which is a better return than most engineering work.

Is topology-aware routing safe? It is a change to your availability posture and must be treated as one. Same-zone preference reduces the effective load-balancing pool and can create hot spots or, with too few replicas per zone, a brief black hole during a deploy, which happened in one rollout. Require a minimum healthy-endpoint count per zone and rehearse the fallback, and then it is a large saving with a small risk.

Does data gravity mean lock-in is permanent? It means the storage location is the real lock-in and the compute API is not. Open table formats make the query engine replaceable while the bytes stay put, which is the achievable half, and the honest position is to decide deliberately where the primary copy lives rather than to pretend the bytes are mobile.

Follow-up Q&A

"When is multi-cloud actually justified?"

When there is a named requirement: a regulator or a customer contract specifying a provider, an acquisition that brought an estate, a service with genuinely no equivalent, or a market where your provider is unavailable. Not for lock-in anxiety with no named risk, and not as resilience against a provider-wide outage, which is rarer than the outages the cross-provider complexity will cause. The cost is continuous rather than one-off, and the largest part of it is that staying portable means avoiding the managed services that make a cloud worth paying for, so you operate your own database, queue, cache and search.

"What should you do instead of multi-cloud?"

A costed exit inventory: every managed service in use, its equivalent on two other providers, and a measured estimate of the migration effort, plus a tested data export in an open format with the duration measured. In one case that took three weeks and produced a number, roughly three engineer-years and five to eight months elapsed, which is what the board actually wanted. It also had an unpredicted second use: a credible, costed exit is the only real leverage in a cloud contract, and the subsequent renewal improved the discount by more than the multi-cloud proposal would have.

"Where does the surprising part of a cloud data-transfer bill come from?"

Not internet egress. In one breakdown of a $47,000 monthly line item, cross-availability-zone traffic was $22,400, NAT gateway data processing was $14,100, inter-region replication was $4,300, and internet egress was $6,200. Three of the four are internal, none of them appear on an architecture diagram, and all are consequences of placement decisions nobody made deliberately: services load-balanced across zones with no locality preference, and object storage accessed through a NAT gateway instead of a private endpoint.

"What is the single highest-return egress fix?"

A gateway or private endpoint for object storage. Traffic to the provider's own storage through a gateway endpoint is free of data processing charges, and through a NAT gateway it costs per gigabyte on top of the egress. In one case 61 percent of NAT gateway traffic was same-region object storage access being routed out to the internet and back and charged for the privilege, and the fix was one Terraform resource and a route table association, worth $8,600 a month, applied in an afternoon.

"How do you reduce cross-zone traffic, and what is the risk?"

Topology-aware routing: prefer a same-zone endpoint when one is healthy and fall back across zones when not. In one case that took the cross-zone share of the two chattiest service pairs from about two thirds to about a tenth, worth $13,900 a month. The risk is that same-zone preference reduces the effective load-balancing pool, so it can create hot spots and, with too few replicas per zone, a brief black hole: one rollout on a service with two replicas per zone sent a zone's traffic nowhere for about twenty seconds during a deploy before fallback engaged. Require a minimum healthy-endpoint count per zone and rehearse the failure, because a cost optimisation that changes routing is a change to your availability posture.

"What is data gravity and what can you do about it?"

Data attracts compute, because moving a petabyte costs both money and weeks while moving compute is nearly free, so the next system gets built next to the data and then the next one. The practical consequence is that lock-in is almost never the compute API, it is the storage location and the systems that accumulated around it. The achievable mitigation is decoupling the query engine from the bytes with an open table format like Iceberg, so the engine is replaceable even though the storage is not, plus a tested export path so the number is known. The rest is deciding deliberately where the primary copy lives.

Common misconceptions

"Kubernetes makes us cloud-portable." It makes the workload API portable. Storage classes, load balancer behaviour, IAM integration and every managed service remain provider-specific.

"Multi-cloud protects against outages." It protects against provider-wide outages, which are rarer than the outages a cross-provider architecture introduces, and only if you have rehearsed the failover, which almost nobody has.

"Egress means the internet bill." Cross-zone traffic and NAT gateway processing are usually larger, and both are internal.

"Terraform makes us portable." It gives you one language and not one semantic. An AWS module and a GCP module for the same concept are not interchangeable.

"Exit egress fees lock us in." The major providers now waive transfer-out charges for customers leaving, following the EU Data Act. The operational transfer charges, which are where the money is, are unaffected.

"Same-zone routing is a free saving." It reduces the load-balancing pool, can create hot spots, and without a minimum healthy-endpoint precondition it can black-hole a zone during a deploy.

Interview delivery note

Say this verbatim: "The surprising part of a data-transfer bill is almost never internet egress. In one case it was forty-seven thousand a month, of which twenty-two was cross-availability-zone and fourteen was NAT gateway processing, and sixty-one percent of the NAT traffic was same-region object storage going out to the internet and back. That fix was a routing table entry." It corrects the intuition and gives the cheapest large fix.

The senior-versus-staff separator is replacing a multi-cloud proposal with a costed exit inventory. A senior engineer argues that multi-cloud is expensive. A staff engineer produces the number: every managed service, its equivalents, a measured migration estimate of roughly three engineer-years and five to eight months, and a tested export that took thirty-one hours for 340 terabytes. That converts a board-level anxiety into an exposure someone can decide about, and in one case it also became the negotiating leverage that improved the next renewal by more than the multi-cloud plan would have saved.

The second signal is treating an egress optimisation as an availability change. Saying "topology-aware routing saved thirteen thousand a month and we rolled it back once, because on a service with two replicas per zone a deploy left a zone with no healthy local endpoint and traffic went nowhere for twenty seconds before fallback" shows you know that changing routing for cost reasons is still changing routing.

The Staff & Team Lead Interview Reference