Step Functions as the orchestrator

Functions (last chapter) run one step. Real agent work is many steps with structure between them: do this, then that, retry on failure, wait for a human, branch on a result. Something has to conduct that, durably, across minutes or days, surviving process restarts. That something is a workflow engine, and on AWS it is Step Functions. This chapter is about when an agent's control flow should be a workflow (explicit, durable, yours) rather than a free loop (model-driven), which is one of the most consequential and most skipped design decisions in the whole field.

Workflow or loop, again

The use-case gallery drew the line and it returns here with infrastructure attached: if you can draw the flowchart in advance, build the flowchart. Step Functions is the flowchart, made executable. You define a state machine (states, transitions, retries, parallel branches, choices) in the Amazon States Language, a published JSON spec, and AWS runs it durably for up to a year per execution, tracking exactly which state each run is in.

The tempting mistake is to make everything a free agent loop because agents are the exciting part. But most real "agentic" work is mostly known control flow with a few genuinely open-ended steps: fetch the data (known), analyze it (judgment, a model call), decide next action (judgment), execute it (known), report (known). Modeling that as a state machine with model-call states where judgment is actually needed beats a free loop on three axes the production bar cares about: it is cheaper (no tokens spent re-deciding the known steps), more auditable (the execution history is the trace), and easier to reason about under failure (each state's retry and error behavior is declared, not emergent). Reserve the free loop for the steps that are genuinely unpredictable, and let the workflow conduct the rest.

The agent loop as a state machine

Even a genuinely agentic loop can live inside a workflow when you want its durability and structure. The loop's cycle maps onto states: a "call model" state, a choice state on the stop reason (done? loop back? call a tool?), a "call tool" state per tool, and a loop-back transition. The event ledger still records the trajectory; Step Functions adds durable execution (a run survives any single component dying, resuming from its last state) and a declared retry policy per state. You give up some of the free loop's flexibility for a lot of operational robustness, which is the right trade for the long-running, must-not-lose-progress agents, and the wrong one for a quick interactive turn.

Standard versus Express is the sizing knob. Standard workflows are durable, auditable, and priced per state transition, good for long-running agent orchestration with human waits. Express workflows are cheap, high-volume, and priced per run, good for short high-frequency event handling. An agent platform typically uses Standard for the per-run orchestration and Express (or plain Lambda) for the fast event plumbing around it.

The human-approval pause, revisited

The approval-gate chapter already named the primitive; here is where it lives. Step Functions' waitForTaskToken is a state that emits a token and stops, holding the run durably (and without billing compute) until something calls back with that token. That is exactly a human-in-the-loop gate: the workflow reaches the gated action, parks, and a human (via a UI, a Slack button behind a Lambda, an email link) posts the decision back with the token, and the run resumes. The reason this belongs in the orchestrator and not the prompt is the reason from Part 8: a workflow state that cannot proceed without a token is a wall, while "ask the human first" in the system prompt is a suggestion an injection can override. The gate is enforced by the engine, outside the model's reach, which is rung four one more time.

Retries, errors, and the poison distinction

Step Functions makes error handling declarative, and the design work is telling two kinds of failure apart, a distinction the failure-semantics chapter drew and this chapter enforces:

• A retryable failure (a model timeout, a throttle, a flaky dependency) gets a retry policy: N attempts, exponential backoff, jitter, then escalate. Declared per state, so a transient blip self-heals without waking anyone.
• A poisoned task (an input that fails every attempt) must not retry forever. After the retries are exhausted, a catch transition routes it to a failure state that dead-letters it for a human, the poison-work discipline the next chapter makes concrete.

The engine gives you both as configuration (Retry and Catch clauses), which means the retry-versus-poison decision is visible in the workflow definition rather than buried in worker code, and visible is auditable. A workflow whose every state declares its retry and catch behavior is a workflow whose failure modes you can review before they happen.

Don't be confused: Step Functions vs the swarm scheduler. They both "coordinate work" and both appeared in fan-out on AWS, so the boundary is worth stating. Step Functions orchestrates the structured, per-run control flow: the states one task moves through, its retries, its human gate. The swarm scheduler manages the continuous, cross-run concerns: the queue of all pending units, priorities, tenant fairness, the shared budget and quota. A big platform runs both, SQS and the scheduler at the front door for intake and fairness, a Step Functions execution per run for its structured phases, which is Hive's actual shape. One conducts a run; the other governs the fleet.

👉 Next: distributed map and fan-out, the Step Functions state that turns one run into ten thousand parallel children, and the managed version of Part 6's swarm fan-out examined from the inside.