LangGraph and LangChain patterns

What these libraries are (plain English)

ShipAI’s rule: hand-roll first. Then learn LangChain / LangGraph so you can read industry code and ship faster without treating the framework as magic.

LangChain helps you glue LLM I/O. LangGraph is the piece that maps cleanly to production orchestration: explicit state, conditional edges, checkpoints, interrupts.

Analogy: LangChain is a parts bin (prompts, retrievers, tool wrappers). LangGraph is the control plane — a programmable state machine that decides which parts run, in what order, and how to resume after a crash or a human “approve/deny.”

One-sentence definition you can defend in an interview

LangGraph is a durable graph runtime over typed state, with conditional edges, checkpoints, and interrupts, used to run LLM agents and workflows you can resume, audit, and evaluate as trajectories.

If you cannot say state, checkpointer, stop condition, and idempotent tools, you are describing a demo notebook — not a product agent runtime.

Interview cue: Frameworks encode patterns you should already understand (ReAct, routers, map-reduce). If you cannot draw the graph on a whiteboard without LangGraph APIs, you are not ready to debug production trajectories.

The problem they solve for LLM apps

A single chat completion is one RPC. Real agents need:

1. Multi-step control flow — model → tools → model → stop
2. Durable state — resume after crash or human approval
3. Branching — route by intent, fan out workers, merge
4. Observability hooks — where to hang traces and budgets
5. Human gates — pause before irreversible side effects

Without a graph runtime you reinvent queues, retries, and “where is the thread state?” in every service. With a graph runtime and no mental model, you get unbounded loops and opaque “Agent stopped” failures.

stateDiagram-v2
  [*] --> Understand
  Understand --> Act: tool_call
  Act --> Understand: tool_result
  Understand --> Respond: final
  Understand --> Stop: max_steps / deny
  Respond --> [*]
  Stop --> [*]

LangChain vs LangGraph vs hand-rolled (do not confuse them)

Ship rule: if the path is known and stable, prefer a workflow. Reach for LangGraph when you need cycles, checkpoints, or branching you will debug for months.

What “Agent stopped” usually means

When Studio or logs say the agent stopped without a useful answer, rank these causes in order:

1. max_steps / budget hit with no final message
2. Tool exception swallowed into an empty observation
3. Conditional edge returned an unexpected label
4. Interrupt waiting for human input nobody resumed
5. Model emitted neither tool_calls nor a clear final

Frameworks do not invent a brain. They make these failure modes addressable if your state and routes are explicit.

Architecture mental model

flowchart TB
  subgraph graph [StateGraph]
    N1[node: call_model]
    N2[node: call_tools]
    N3[node: human_gate]
    N1 -->|conditional| N2
    N1 -->|final| Out[END]
    N2 --> N1
    N1 -->|needs_approval| N3
    N3 --> N2
  end
  CP[(Checkpointer)] <--> graph
  App[Your API] --> graph

What the checkpointer actually buys you

Without persistence, a process kill mid-tool means “start over” and often double side effects. With a checkpointer:

• Each super-step writes state keyed by thread_id
• You can resume after deploy, crash, or human sleep
• You can time-travel to an earlier checkpoint for debug (carefully — secrets may live there)

sequenceDiagram
  participant API as Your API
  participant G as StateGraph
  participant CP as Checkpointer
  participant T as Tools
  API->>G: invoke(thread_id)
  G->>CP: load / save state
  G->>T: tool call
  Note over G: crash here without CP = lost + maybe double-charge
  G->>CP: checkpoint after node
  API->>G: resume(thread_id)
  G->>CP: load

Reducers: the silent correctness bug

When two parallel nodes update the same field, LangGraph needs a reducer. Classic pattern:

Anti-pattern: mutable in-place edits of a shared list without a reducer. Replays and fan-out then disagree about history.

How it fits an LLM product

Typical placement:

LangGraph does not replace your gateway, auth, or evals. It replaces ad-hoc while True orchestration once the loop is proven.

flowchart TB
  FE[Chat / ticket UI] --> GW[Gateway]
  GW --> Auth[Auth + tenant]
  Auth --> AG[Agent service / LangGraph]
  AG --> LLM[Model provider]
  AG --> Pol[Tool policy]
  Pol --> MCP[MCP / HTTP tools]
  Pol --> RAG[Retriever]
  AG --> CP[(Checkpoints)]
  AG --> Tr[Trajectory + OTel]

Streaming to the user vs streaming inside the graph

Two different streams:

1. Token stream from the model node to the UI (UX)
2. Event stream of node starts/ends, tool calls, interrupts (ops)

Ship both. Users care about tokens; on-call cares about “stuck in call_tools for 40s.” Pair token streaming with Serving and streaming and span events with OpenTelemetry for LLMs.

Minimal graph shape (code)

# Shape only — APIs evolve; pin versions in your repo
from typing import TypedDict, Annotated, Literal
import operator

class AgentState(TypedDict):
    messages: Annotated[list, operator.add]
    steps: int
    tenant_id: str
    pending_approval: dict | None

def call_model(state: AgentState) -> dict:
    # LLM with tools bound → append AIMessage (maybe with tool_calls)
    ...

def call_tools(state: AgentState) -> dict:
    # Execute tool_calls; append ToolMessages
    ...

def human_gate(state: AgentState) -> dict:
    # interrupt() before mutating tools; resume with approval payload
    ...

def route(state: AgentState) -> Literal["tools", "end", "stop", "approve"]:
    if state["steps"] >= 8:
        return "stop"
    last = state["messages"][-1]
    if getattr(last, "tool_calls", None):
        if needs_hitl(last):
            return "approve"
        return "tools"
    return "end"

# builder.add_node("model", call_model)
# builder.add_node("tools", call_tools)
# builder.add_node("approve", human_gate)
# builder.add_conditional_edges("model", route, {...})
# graph = builder.compile(checkpointer=MemorySaver())
# graph.invoke(
#   {"messages": [...], "steps": 0, "tenant_id": "...", "pending_approval": None},
#   config={"configurable": {"thread_id": "tenant:user:session"}},
# )

Ship rule: always put steps (or token budget) in state and enforce it in route — frameworks will not save you from infinite tool loops.

LCEL for linear pipelines (when LangGraph is overkill)

# Shape — retrieve → prompt → LLM → parse
# chain = retriever | prompt | llm | parser
# Prefer this when there is no cycle and no HITL.

Use LCEL / simple chains for fixed RAG Q&A. Graduate to LangGraph when you add tool loops, routers, or interrupts.

Thread IDs that survive multi-tenant products

Prefer tenant_id:user_id:conversation_id (or opaque server-issued ids mapped in your DB). AuthZ happens in the gateway and tool layer — state should carry tenant_id as a claim you already verified, not as something the model “remembers.”

Patterns worth stealing

1. Router — classify intent → specialist subgraph (billing vs search vs code)
2. Orchestrator–worker — plan, fan out parallel workers, reduce
3. HITL — interrupt before refunds, sends, deletes
4. Map-reduce over docs — retrieve many, summarize per shard, merge
5. Supervisor — one LLM assigns work to tool-specialist agents
6. Retry with backoff node — isolated flaky-tool handling without polluting the main loop
7. Critic / verifier pass — second model checks citations or policy before final

flowchart LR
  U[User] --> R[Router]
  R --> A[Subgraph A]
  R --> B[Subgraph B]
  A --> M[Merge / respond]
  B --> M

Walkthrough: support refund with HITL

1. User: “Refund order A-1042.”
2. call_model → tool get_order → eligible.
3. Graph edges to human_gate (interrupt) before issue_refund.
4. Operator approves in your UI; API resumes same thread_id.
5. Tool runs once; checkpoint records success; trajectory is eval-ready.

Without interrupt + idempotency keys, retries after timeout double-refund.

Walkthrough: research map-reduce

1. Router picks “research” subgraph.
2. Fan-out: N workers summarize N sources in parallel.
3. Reduce node merges notes under a token budget.
4. Final model writes cited brief.

Same pattern as classic MapReduce — the LLM is just the mapper/reducer function.

Walkthrough: agentic RAG inside the graph

1. Model decides whether to retrieve.
2. Retriever tool returns top-k with doc_id + spans.
3. Model answers with citations; if weak evidence, retrieve again (bounded by steps).
4. Stop when answer + citations meet schema, or budget expires.

Do not bake “always retrieve once” into a chain if the product needs multi-hop. Do not leave retrieve unbounded either — that is how context and cost explode. Pair with RAG building blocks and LlamaIndex when the retrieval stack is the hard part.

State design that survives production

Anti-patterns: stuffing full PDF bytes into state; storing raw API keys in checkpoints; mutable globals outside state (unreplayable); letting the model invent tenant_id.

Context hygiene inside long runs

Context engineering is not optional once graphs loop — see Context engineering.

How it works end-to-end (request lifecycle)

sequenceDiagram
  participant U as User / UI
  participant GW as Gateway
  participant AG as Agent service
  participant G as StateGraph
  participant CP as Checkpointer
  participant LLM as Model API
  participant T as Tools / MCP
  U->>GW: POST /agent (auth, tenant)
  GW->>AG: start or resume(thread_id)
  AG->>CP: load checkpoint
  AG->>G: invoke / stream
  loop until stop / interrupt
    G->>LLM: messages + tools
    LLM-->>G: tool_calls or final
    G->>T: execute (policy)
    T-->>G: observation
    G->>CP: save
  end
  alt interrupt
    AG-->>U: needs_approval payload
  else done
    AG-->>U: answer + citations
  end
  AG->>AG: write trajectory JSONL + OTel spans

Idempotency contract for mutating tools

Every write tool should accept a client-generated idempotency key stored on success:

1. Before side effect: check key → if done, return prior result
2. Execute once
3. Persist key + result in the same transaction as the business write when possible

Checkpoint restore + HTTP retry without this contract is how agents double-email and double-charge.

Alternatives (when not to use LangGraph)

LangChain LCEL alone is fine for linear pipelines. Reach for LangGraph when you need cycles, persistence, or branching.

Framework vs workflow engine

Many products use both: LangGraph for the agent brain, Temporal for “wait three days for a human form.”

Production gotchas

• Unbounded loops — missing max_steps / budget
• Context blow-ups — appending full tool dumps forever; summarize or drop
• Swallowed tool errors — surface errors into state; don’t collapse to “Agent stopped”
• Checkpoints with secrets — redact API keys, tokens, PII before persist
• Non-idempotent tools — retries after checkpoint restore double-charge
• Version skew — graph code changed but old threads resume into new nodes
• Studio-only debugging — always keep a trajectory log you can replay without UI
• Over-graphing — ten nodes for a three-step chain; prefer LCEL until cycles appear
• Parallel fan-out without reducers — lost or duplicated state updates
• Trusting model-supplied tenant fields — authZ bypass waiting to happen

Always log a trajectory (messages + tool I/O + latencies) you could replay without Studio UI. Pair with OpenTelemetry for LLMs.

flowchart LR
  Fail[Failure] --> Cause{Root cause?}
  Cause -->|No max_steps| Fix1[Add steps + budget in route]
  Cause -->|Huge tool dump| Fix2[Truncate + summarize]
  Cause -->|Double side effect| Fix3[Idempotency + interrupt]
  Cause -->|Resume into new graph| Fix4[Version graph + migrate threads]

How to evaluate graph agents

Freeze a golden task set (goal → expected tools → final assertion). Score trajectories, not just final text — same lesson as evals fundamentals.

Offline vs online evals

Never promote a graph because Studio demos looked smooth.

Debugging trajectories like production engineers

Ship rule: store a framework-agnostic JSONL trajectory (step, node, latency_ms, tool, ok, tokens). Studio is optional; replay is not.

Debugging playbook (first hour)

1. Pull trajectory for thread_id; find last successful node.
2. Diff route decision vs expected.
3. Re-run tools with the same args outside the graph (isolate flaky deps).
4. Check checkpoint size / secrets / graph_version.
5. Reproduce with pinned model + temperature 0 if nondeterminism confuses you.
6. Only then change prompts — most “model bugs” are state/route bugs.

Checkpointer backends (practical)

Whatever you pick: encryption at rest for PII, retention policy, and a plan for “delete user → delete threads.”

Retention and compliance

Treat checkpoints like a database of conversations, not temporary cache — because that is what they become.

Subgraphs and skills packaging

Large orgs split graphs the way they split services:

• Billing subgraph with refund interrupt
• Search subgraph with retriever tool only
• Code subgraph with sandboxed shell

The parent graph routes; child graphs own specialized state slices. Version subgraphs independently so a search change does not invalidate billing threads mid-flight.

flowchart TB
  Parent[Parent router graph] --> Bill[Billing subgraph]
  Parent --> Search[Search subgraph]
  Parent --> Code[Code subgraph]
  Bill --> HITL[Refund interrupt]
  Search --> Ret[Retriever tools]
  Code --> Sand[Sandbox tools]

Multi-agent preview (honest scope)

Supervisor patterns in LangGraph are still one product concern: who owns stop conditions, shared memory, and tool allowlists. See Multi-agent orchestration before spinning five agents that share a credit card tool.

Observability: what to log every run

Emit OpenTelemetry spans per node; attach prompt_version and retriever_version when RAG is involved.

Security and blast radius (non-optional)

Companion reads: Guardrails and safety.

Memory, cost, and sizing intuition

Ballpark: if golden tasks average 6 steps at 3k tokens/step, 100k tasks/month ≈ 1.8B tokens before caching — budget before you celebrate “agents everywhere.”

How to practice

1. Hand-roll a 3-tool ReAct loop (Agents and ReAct).
2. Port the same state machine to a LangGraph StateGraph.
3. Add a checkpointer + thread_id; kill the process mid-run and resume.
4. Add an interrupt before one irreversible tool.
5. Log trajectories; score against a 10-task golden set.
6. Break idempotency on purpose; confirm double side effects; then fix.

Micro-project

Port a hand-rolled agent with tools search_docs, get_order, create_ticket into a LangGraph graph that:

1. Enforces max_steps=8 in the router
2. Checkpoints after every node
3. Interrupts before create_ticket
4. Emits a trajectory JSONL line per step
5. Records graph_version on the thread

Acceptance: resume after kill still issues at most one ticket for the same idempotency key.

Interview whiteboard: draw this first

When someone says “we use LangGraph,” interviewers want this sketch in under two minutes:

1. State fields (messages, steps, tenant, graph_version)
2. Nodes (model, tools, HITL)
3. Conditional edges with an explicit stop
4. Checkpointer keyed by thread_id
5. Trajectory log outside the framework UI

If you skip stop conditions or idempotency, you have a demo graph — not a product runtime.

Interview prompts you should be able to answer

1. When is LCEL enough vs LangGraph?
2. What does a checkpointer store, and what must you redact?
3. How do you prevent double refunds on resume?
4. How do you version a graph so old threads do not crash?
5. What metrics prove an agent got better, not just wordier?

Common interview traps

Anti-patterns (LangGraph edition)

• Wrapping a single LLM call in a five-node graph
• Putting PDF bytes and API keys in state
• Fan-out without reducers
• HITL that emails a human with no resume API
• One mega-graph for billing + code + search with shared write tools
• Evaluating only final prose, never tool choice

Tradeoffs summary

Checklist before you ship a graph

• max_steps / token budget enforced in router
• Mutating tools behind interrupt + idempotency
• Checkpoints redact secrets; encryption + retention set
• Trajectory export works without Studio
• Golden task suite scores tool choice + success
• Graph version recorded on every thread
• tenant_id from auth, never from the model
• OTel spans on nodes + tools
• Resume tested after process kill and after deploy

End-to-end lab checklist (do this once)

1. Hand-roll ReAct with three tools.
2. Port to StateGraph with identical tool schemas.
3. Add MemorySaver; kill mid-tool; resume.
4. Swap to Postgres checkpointer; repeat.
5. Add interrupt + UI approve path.
6. Export JSONL; score ten golden tasks.
7. Rename a node; document migration strategy for open threads.

What “good” looks like in a design doc

A short design should name: state schema, stop policy, tool allowlist, checkpointer backend, HITL surfaces, eval suite, and failure modes. If it only says “we’ll use LangGraph,” it is not a design.

Glossary

Related

Core Concept: Agents and ReAct. Guided Build real AI agents, Skills, MCP, context engineering, and Agentic workflows & multi-agent. Key tech: Model Context Protocol when tools leave process boundaries. Advanced: Multi-agent orchestration, Guardrails and safety, Context engineering. Observability: OpenTelemetry for LLMs.