Planning vs reactive

Some tasks need a plan object; some need tight react loops

Reactive ReAct decides the next tool call based on the latest observation — excellent for exploratory tasks with uncertain steps. Plan-then-execute asks the model to emit a structured plan first, then an executor walks the steps — better when decomposition is stable, tools are expensive, or humans want previewability.

Neither wins always. This lesson implements both on the same task and compares trajectories, step count, success rate, and plan failure modes.

Callout — measure on identical tasks: Planner overhead hurts simple one-tool queries — know when to route reactive instead.

Planner → executor architecture

User goal
    ↓
Planner LLM → Plan JSON (steps with tool + args sketch)
    ↓
Validator (schema, policy)
    ↓
Executor loop (step by step, optional replan trigger)
    ↓
Final answer

Plan schema example:

{
  "goal": "Summarize open tickets for customer 442",
  "steps": [
    {"id": 1, "tool": "get_customer", "args": {"id": 442}},
    {"id": 2, "tool": "list_tickets", "args": {"customer_id": 442, "status": "open"}},
    {"id": 3, "tool": "summarize", "args": {"ticket_ids": "$step2.ids"}}
  ]
}

Executor resolves $step2.ids references from prior results — simple variable substitution, not free-form model rewrite mid-flight.

When planning helps

• Multi-step workflows with known structure (onboarding checklist, ETL-style fetches).
• Expensive tools where blind retry is costly.
• Human review of plan before execution (ties to HITL).
• Compliance audit trail requiring upfront intent.

When reactive wins

• Single-hop lookup with unknown identifier format.
• Environment feedback changes strategy (404 → search alternate path).
• Debugging/interactive sessions where plan would stale immediately.

Router pattern (optional stretch): classifier chooses planner vs reactive per query — document heuristic in README.

Plan quality failures

Log these explicitly in plan_failures.jsonl:

Callout — replan is not cheating: Production systems replan; log replan events separately from initial plan quality.

Partial plan execution

Not all steps must succeed before partial value delivery. Options:

Fail-fast: Stop on first tool error — simple, good for transactional workflows.

Best-effort: Continue with null placeholders; final summarize step notes missing data — good for reporting.

Replan: On step failure, call planner with error context — powerful, adds latency.

Log which policy you used; compare success rates in COMPARISON.md.

Human-readable plan preview

Before execution in demo CLI:

Plan (3 steps):
  1. get_customer(id=442)
  2. list_tickets(customer_id=442)
  3. summarize(ticket_ids=...)
Execute? [y/n]

Optional HITL bridge before lesson 7.5 — user approves plan, not individual tools.

Stochastic planner variance

Run planner twice on same input with temperature > 0 — plans may differ. For eval:

• Use temperature 0 for planner when measuring plan quality
• Or score plan success rate over 3 samples (pass if any plan succeeds)

High variance plans suggest task needs reactive mode or richer plan schema constraints.

Executor error messages back to planner

When replanning, pass structured error:

{"failed_step": 2, "tool": "list_tickets", "error": "customer_id not found"}

Planner prompt: "Revise plan given failure; do not repeat failed args verbatim."

Without structured errors, replan loops re-hallucinate same bad customer id.

When to route reactive vs planner

Simple router heuristic for portfolio:

def mode_for(query: str) -> str:
    if len(query.split()) < 12 and " and " not in query.lower():
        return "reactive"
    return "planner"

Log router decisions; refine with eval tags instead of premature ML classifiers.

Plan schema validation

Validate plan JSON with Pydantic before executor starts:

class PlanStep(BaseModel):
    id: int
    tool: str
    args: dict

class Plan(BaseModel):
    goal: str
    steps: list[PlanStep] = Field(max_length=12)

Reject unknown tool names at validation — executor never sees hallucinated tools.

Comparison methodology

Task suite: 10–15 scenarios from your tool domain.

Metrics:

• Success rate (task rubric pass/fail)
• Total tool calls
• Wall clock latency
• Plan revisions count (planner path)
• Human-readable failure notes

Table in COMPARISON.md:

Expect planner wins on multi-fetch reports; reactive wins on "what is ticket T-99 status?"

Implementation notes

Share tool registry between modes — only orchestration differs.

Planner prompt: output JSON only, no execution.

Executor: deterministic; model may format final summarize step unless that step is pure code.

Keep trajectories for both modes under runs/reactive/ and runs/planner/.

Engineering problem (staff framing)

Plans help multi-step goals; reactive loops adapt. Wrong mode wastes tokens or ignores new evidence.

Diagram — Plan then act vs react

flowchart LR
  subgraph Plan
    G[Goal] --> P[Plan] --> E1[Exec steps]
  end
  subgraph React
    O[Observe] --> A[Act] --> O
  end

Precise definitions & mental model

Planner/executor, replanning triggers, DAG workflows vs free ReAct.

Tradeoffs — when to use what

Upfront plan (clear UX) vs reactive (robust to tool noise).

Failure modes (interview + on-call)

Brittle long plans; reactive thrashing without progress metric.

Production & OSS practices

Progress heuristics; force replan on repeated tool errors.

Deep dive (FAANG / OSS bar)

Push «planning-vs-reactive» past tutorial depth: write the interface contract (inputs/outputs/invariants), list three measurable metrics, and name two degrade modes if the happy path fails. Add a short threat note: what an attacker or noisy tool result could do, and which layer catches it (schema, policy, HITL, or eval gate).

flowchart LR
  Contract[Interface contract] --> Metrics
  Metrics --> Degrade[Degrade modes]
  Degrade --> Threat[Threat + control]

Micro-project: Planner→executor

In m7/planner_executor/:

1. Planner producing validated plan JSON.
2. Executor walking steps with reference substitution.
3. Same 10 tasks run reactive (7.1 loop) and planner paths.
4. COMPARISON.md + plan_failures.jsonl with ≥3 logged plan failures.

Checklist

• Plan schema validated before execution
• Fair comparison on same tasks
• Plan failures categorized
• README states when you would route each mode