Fan-out on AWS: three ways to run the same fleet

The simulator is four pieces of plain code. Running the same design against real infrastructure means choosing, for each piece, whether AWS supplies it or you do, and the choice comes in three coherent bundles. This chapter walks the three, gives the decision table, and then solves the problem the simulator waved away: five hundred workers on different machines sharing one token quota without either storming it or wasting it.

Bundle one: Step Functions distributed map

The most managed shape. Your work units land in S3 (one object or one manifest line each); a distributed map state fans them out, up to 10,000 concurrent child executions, each child running one unit (a Lambda-hosted agent turn sequence, or a Fargate task for longer units); failure thresholds stop a run that is going wrong; results aggregate back to S3.

Decoded against the four pieces: the queue and dispatcher are inside the map state (AWS's), the workers are yours, and the two distinctively agentic pieces (governor and ledger) have no managed equivalent, so they live inside your worker code or in front of the map. What the bundle buys is significant: redrive is checkpoint-and-resume as a button (re-run only failed children), and the failure-threshold knob is a crude but real blast-radius cap. What it costs is control: no priorities, no continuous intake (a map run is batch-shaped: it starts, fans, ends), and mid-run steering limited to what you wired in.

Best fit: batch-shaped fleet runs over a known input set, which is Capstone A's overnight audit almost exactly.

Bundle two: SQS plus a worker fleet

The most sovereign shape, and Hive's default. Units flow into SQS; a fleet of Fargate tasks (or Lambda, for sub-15-minute units) long polls the queue; your scheduler code does admission against the ledger, your governor shapes model calls, and everything the simulator did in one process now runs as a small service.

You inherit SQS's honest contract: at-least-once delivery, visibility timeouts (tune them to agent-scale unit durations, minutes not milliseconds, or healthy workers will have their units re-delivered mid-run), and dead-letter queues as the poison-work parking lot. What the bundle buys is everything the map lacks: continuous intake, priorities (multiple queues or priority lanes), tenant fairness, mid-run control, and a natural home for the ledger and governor as first-class components. What it costs is that you now operate a scheduler.

Best fit: continuous, multi-tenant, budget-governed agent platforms, which is to say the control plane of the reference architecture.

Bundle three: AgentCore Runtime sessions

The most agent-native shape. Each unit becomes a Runtime session (Chapter 16): dispatched by a thin coordinator (a Lambda draining SQS, or the map's children invoking sessions), isolated per session in its microVM, billed CPU-active only. The fleet stops being infrastructure you size and becomes sessions you count.

The distinctive economics follow from the billing model: units that wait (on slow tools, on approvals, on other agents) cost nearly nothing while waiting, so this bundle wins exactly where the fleet economics punished always-on workers: sparse, long, interaction-heavy units. Saturated overnight number crunching still favors Fargate's flat rate. Governor and ledger remain yours in every case; no vendor knows your budget.

The decision table

The distributed governor

Now the promised hard problem. The simulator's governor was a variable; five hundred workers on different machines cannot share a variable, and the naive fix (divide the quota by N, give each worker a slice) quietly destroys the burst capacity the bucket exists to provide: worker 17's idle slice cannot serve worker 292's burst. Three real designs, in ascending sophistication:

• Central counter. The bucket is a row: DynamoDB with conditional writes (the state plane's concurrency primitive) or ElastiCache. Workers draw tokens with an atomic decrement before each model call. Correct and simple; at fleet scale the row becomes hot, so nobody ships the naive version, everybody ships the next one.
• Batched draws. Workers draw allowances, not tokens: 60,000 tokens per request to the counter, spent locally, refreshed when low. Contention drops by the batch factor; the price is slack (up to one batch per worker of quota can sit unused at any moment), so batch size becomes a tuning knob between contention and utilization. This is the design the capstones run.
• Adaptive concurrency. Skip explicit token accounting; treat throttle signals as backpressure and adjust each worker's concurrency window, AIMD-style like TCP. Elegant, self-tuning, and it discovers the limit by brushing it, which reintroduces a polite version of the storms Chapter 14 abolished. Reasonable as a complement, risky as the only governor.

The composite that works: batched draws against a central counter for the steady state, a small headroom reservation (the bystander lesson), and admission control at the dispatcher so the queue, not the governor, absorbs demand spikes. Every piece of that sentence is a line item in the simulator you have already run.

Don't be confused: distributed map vs the Parallel state. Step Functions has both. Parallel runs a fixed set of named branches drawn in the workflow definition: do these four different things at once. Distributed map runs one branch over a dataset: do this same thing ten thousand times. Fleet fan-out is map-shaped; pipelines with simultaneous stages are Parallel-shaped; a workflow can nest both, and the capstones do.

👉 Next: failure semantics, where the fleet learns to be killed: retries versus poison, leases, partial results, and the lab that murders the simulator mid-run and audits exactly what the crash cost.