What agentic workflow demos never show

Most agentic workflow demos end at the happy path: the agent receives a task, calls a tool, returns a result. Pretty slides, clean flow. What never appears: the agent calls a tool with bad input in step 4 of 6, the LLM returns a hallucinated action in step 3, a downstream service times out after 15 minutes, or worse — the agent runs correctly but commits a side effect it should never have committed.

These are real governance problems in production. The answer is not sprinkling try-catch everywhere in code. It is designing a state machine with retry semantics, human approval gates, idempotency contracts, and compensation flows from day one.

EventBridge / SQSStep FunctionsAgentCoreLambda WorkerHuman Approverstart executioninvoke agent steptool call (async)result + compensationDataemit task token (pause)sendTaskSuccess(token)continue next statefinal result
Wait for Task Token: state machine pauses, zero compute, until human sends callback

Why state machine, not code

The common pattern when first building an agentic pipeline: orchestrate everything in code. Call Lambda A, get result, call Lambda B conditionally. Simple when small, but it fractures in non-obvious ways:

Timeout: Lambda execution limit is 15 minutes. Long pipelines (voice transcription + diarization + summarization) with variable-length inputs — 2-hour audio easily exceeds the limit.

No history: When a production incident happens at 2 a.m., there is no way to know where the execution reached, which step failed, or with what input.

Scattered retry logic: Every step writes its own retry, inconsistent, no standard exponential backoff.

Human approval: Implementing human-in-the-loop without burning Lambda idle time while waiting requires special architecture — most naive implementations poll DynamoDB and waste money.

AWS Step Functions solves all of this at the infrastructure level. The state machine is a definition (JSON/YAML), not running code. Built-in visual debugger. 90 days of execution history. Retry semantics defined once at the state level.

Code Orchestration
  • Retry logic written differently in every Lambda — no consistent BackoffRate or MaxAttempts
  • Lambda timeout (15 min) blocks long-running steps with variable-length inputs
  • No execution history: debug production incidents by adding log statements
  • Human approval requires Lambda polling DynamoDB — billed during idle wait time
Step Functions
  • MaxAttempts + BackoffRate defined once at state level — uniform across all steps
  • Wait for Task Token: pause state machine for hours/days — zero compute billing
  • 90-day execution history with visual debugger — replay any execution to any state
  • Catch → CompensationFlow: native Saga routing with full execution context preserved

Wait for Task Token is the killer feature

Step Functions can emit an opaque token and pause the state machine immediately — no Lambda running, no compute billed. The token is the contract: whoever holds it can resume the execution later. Human approves the next day? The state machine resumes in milliseconds after sendTaskSuccess(token). Lambda timeout is irrelevant. Heartbeat timeout handles abandonment.

This is how you do human-in-the-loop in production without paying for idle compute.

Wait for Task Token ≠ polling

Step Functions emits an opaque token and pauses immediately — no Lambda running, no compute billed. The token is the contract: whoever holds it can resume the execution. Human approves next day? State machine resumes in milliseconds after sendTaskSuccess(token). Lambda timeout is irrelevant. Heartbeat timeout handles abandonment.

AgentCore and Step Functions are complementary, not alternatives

AgentCore governs per-agent concerns: tool routing by intent, multi-turn memory (episodic, semantic, procedural), and fine-grained identity/permissions.

Step Functions governs the workflow: state transitions, error routing, retry/backoff policies, and compensation (Saga pattern).

You stack them. AgentCore steps become tasks inside the Step Functions definition.

  1. Runtime

    Tool routing by intent, multi-turn conversation state, retry on tool call failure. Agent developer writes intent, not routing logic.

  2. Memory

    Episodic (conversation history), semantic (extracted facts), procedural (learned patterns). Persists across sessions without application code managing state.

  3. Identity

    Agent-level permissions: agent X, tool Y, tenant Z. Finer-grained than IAM roles — authorization at the agent action level, not function level.

  4. Gateway

    Unified API for Lambda, API Gateway, MCP servers. Tool implementation can migrate without touching agent logic — clean abstraction boundary.

Idempotency and compensation are non-negotiable

Every Lambda step must be idempotent using a key derived from executionId + stepId. When Step Functions retries (for any reason), the step sees the same key, finds the prior result (or in-progress marker), and returns safely without duplicate side effects.

Compensation is domain-specific. When a step fails, Step Functions can route to a CompensationFlow. But you must design what "undo" means for your domain — write compensationData alongside the result in each step, then execute undos in reverse order, handling partial failures.

No managed service will write your business rollback logic for you.

  1. 1Check DynamoDB keyQuery executionId + stepName — unique per Step Functions execution per state
  2. 2Already completed?Return cached result — safe regardless of how many times Step Functions retried
  3. 3In progress?Concurrent retry detected — throw CONCURRENT_EXECUTION, Step Functions retries after interval
  4. 4Execute + compensateRun step logic, write compensationData for Saga rollback alongside result
  5. 5Mark completedUpdate key to status:completed with TTL 24h — stale keys auto-expire, no cleanup job needed