Workflow patterns

Agentic workflows are graphs, not bigger prompts

A single ReAct loop suffices for narrow tasks. Product features — research reports, multi-source triage, code migration pipelines — need graphs of roles, branches, and handoffs. Agentic workflow patterns are reusable graph shapes: who decides, who executes, how state flows, and where the run stops.

The mistake to avoid is treating "multi-agent" as permission to spawn five chatty LLM personas without a spec. Patterns exist to bound coordination cost: fewer ambiguous handoffs, clearer failure domains, testable stage boundaries.

This lesson implements two patterns and compares how each fails when tools error, models drift, or workers disagree.

Pattern 1: Router (dispatch)

Shape: One classifier node reads the user request and routes to exactly one specialized handler (or declines).

User → Router → {Support | Billing | Technical} → Response

When to use:

• Intents are mostly separable with low overlap.
• Specialists do not need each other's full reasoning trace.
• Latency budget favors one main execution path.

Implementation sketch:

• Router can be rules, embeddings, or a small LLM call returning an enum.
• Pass forward only { intent, user_message, session_id } plus shared read-only context.
• Log route_decision and confidence.

Failure modes:

• Misroute — Billing question lands in Support; wrong tools loaded. Mitigate with router eval set and human override path.
• Overconfidence — Router always picks default bucket. Monitor entropy / score spread.
• Stale routes — New product area has no bucket; add explicit UNKNOWN → ask clarifying question.

Router is the cheapest multi-step pattern. Start here before orchestrator complexity.

Pattern 2: Orchestrator–worker

Shape: A planner decomposes the task, assigns subtasks to workers, synthesizes results.

User → Orchestrator → Worker A (research)
                    → Worker B (draft)
                    → Orchestrator (merge) → Response

When to use:

• Task naturally splits into parallel or sequential subtasks with clear outputs.
• Subtasks need different tools or skills (search vs. writing vs. calculation).
• You want one place to enforce global stop conditions and budgets.

Worker contract: Each worker receives a task packet:

{
  "objective": "Summarize refund policy for EU customers",
  "constraints": ["cite sources", "max 300 words"],
  "tools_allowed": ["policy_search"],
  "output_schema": {"summary": "string", "citations": ["string"]}
}

Workers return structured output, not chat prose, when possible — orchestrator merge becomes deterministic.

Failure modes:

• Fan-out explosion — Orchestrator spawns too many workers. Cap max_workers and require justification field per spawn.
• Merge loss — Orchestrator drops worker caveats. Require workers to emit confidence and open_questions.
• Circular replanning — Orchestrator never declares done. Hard max_replan and explicit finish action.

Alternative: Parallel / debate (optional third pattern)

For eval-heavy or high-stakes reasoning, run parallel workers with a judge merge:

User → Worker 1 ─┐
     → Worker 2 ─┼→ Judge → Response
     → Worker 3 ─┘

Use when diversity reduces error (legal interpretation, ambiguous specs), not when workers would duplicate identical tool calls. Cost and latency multiply — budget accordingly.

Debate failures: false consensus (judge picks longest answer), rubber-stamping (workers correlate errors). Mitigate with diverse prompts/tools per worker and component evals on workers separately.

Comparing patterns

Callout — Pattern ≠ framework: LangGraph, Temporal, or plain Python state machines can all implement these shapes. ShipAI expects you to hand-roll one orchestrator loop first so you feel where frameworks hide complexity.

When to combine patterns

Production systems nest patterns: router picks domain → orchestrator runs internal pipeline → single worker calls tools. Avoid deep nesting without span logging — debug time grows with depth.

Document pattern boundaries in your module README: which nodes are LLM calls vs. deterministic code vs. human approval gates.

Engineering problem (staff framing)

Multi-step AI work needs workflow patterns (pipeline, router, supervisor, map-reduce) with explicit state — not ad-hoc threads.

Diagram — Common workflow patterns

flowchart TD
  In[Input] --> R{Router}
  R --> A[Agent A]
  R --> B[Agent B]
  A --> Join[Join / reduce]
  B --> Join --> Out

Precise definitions & mental model

Pipeline, parallel fan-out, supervisor, hierarchical agents; deterministic edges vs LLM routers.

Tradeoffs — when to use what

Failure modes (interview + on-call)

Hidden shared mutable state; no idempotency; infinite supervisor loops.

Production & OSS practices

Draw the graph in design review; SLOs per node; dead-letter queues.

Deep dive (FAANG / OSS bar)

Deterministic edges beat clever routers (initially)

Start with a fixed pipeline for RAG: rewrite → retrieve → rerank → generate → cite. Replace a stage with an LLM router only when you have metrics showing need. Supervisors are powerful and expensive.

Idempotency keys

Fan-out/fan-in workflows retry. Every mutating node needs an idempotency key derived from (run_id, node, input_hash).

Micro-project: Implement 2 patterns

In your portfolio workflow folder:

1. Implement router dispatching to two handlers with different tool sets (can be toy domains).
2. Implement orchestrator–worker with one orchestrator and two workers (sequential or parallel).
3. Write patterns.md comparing failure modes you observed in manual chaos tests (misroute, worker timeout, bad merge).
4. Add decision guide: "We would pick router when… / orchestrator when…"
5. Log graph events: node_enter, node_exit, route, worker_output_schema_valid.

Acceptance: same sample user request can be handled by both patterns in demo mode; docs explain tradeoffs with your own trace IDs.

Checklist

• Router pattern runnable with logged route decisions
• Orchestrator–worker with structured worker packets
• patterns.md with failure comparison from real runs
• Decision guide for when to use each pattern
• Module README links to traces for both patterns