Evals fundamentals

Why evals before demos

LLM demos lie. Temperature, prompt drift, index skew, and cherry-picked questions make anything look smart for five minutes.

Evals are the release gate: fixed cases, scored outputs, regression detection when you change prompt, model, chunker, or tools. ShipAI rule: no milestone “done” without an eval artifact.

flowchart LR
  Gold[Golden set] --> Offline[Offline CI eval]
  Offline --> Ship{Pass?}
  Ship -->|no| Fix[Fix prompt/RAG/agent]
  Ship -->|yes| Prod[Deploy]
  Prod --> Online[Online traces + canaries]
  Online --> Gold

Mental model: what you are measuring

An eval needs four parts:

1. Dataset — inputs (and optional context/tools state)
2. System under test — prompt + model + retrieval + agent config (versioned)
3. Scorer — exact match, rubric, retrieval metric, policy check, LLM-judge
4. Gate — thresholds that block merge/deploy

Without (4), you have dashboards, not engineering.

Types you will actually use

Component vs E2E: score retrieval and final answer separately. A pretty answer with wrong chunks is still a fail — and you need to know which stage broke.

flowchart TD
  Q[Query] --> Ret[Retrieval]
  Ret --> Gen[Generation]
  Ret --> M1[recall@k]
  Gen --> M2[answer score]
  Gen --> M3[citation validity]
  M1 --> Gate[Release gate]
  M2 --> Gate
  M3 --> Gate

Offline golden sets (how to build them)

Start small and mean:

1. Pick one feature (FAQ bot, extractor, agent).
2. Write 20–50 cases: happy path, edge cases, adversarial, “should refuse.”
3. For RAG: label expected chunk IDs or must-include facts.
4. For agents: label expected tool sequence constraints (must call X; must not call Y).
5. Freeze the set in git; version alongside prompts.

Hard negatives matter more than more happy paths — queries that used to fool you.

LLM-as-judge (use carefully)

Judges scale rubrics (“faithful?”, “tone OK?”) but:

• Can be biased toward verbose answers
• Need calibration against human labels on a slice
• Must see the same evidence the system had for faithfulness checks
• Belong behind a versioned judge prompt — treat like production code

Prefer deterministic checks when possible (JSON schema, citation ID ∈ retrieved set, regex for required fields).

Online evals and feedback loops

Offline gates catch regressions before ship. Online catches reality:

• Implicit signals: retries, abandon, rephrase, escalation to human
• Explicit: thumbs, “wrong citation”
• Canaries: shadow new prompt/model on % traffic; compare scores
• Trace sampling: store inputs/outputs for periodic human review (watch PII — see Privacy)

Feed failures back into the golden set weekly or you will re-learn the same bug forever.

How to wire evals into a build

PR opens → run offline suite → fail CI if gate missed
deploy → canary → promote
weekly → mine prod fails → add goldens

Pair with observability (OpenTelemetry, LangSmith, etc.) so a failed case is replayable.

Tools today (2025–2026)

Guided Evals, guardrails, safety is the full lab; Advanced Concepts covers guardrails alongside evals.

Failure modes

Tradeoffs

• More cases vs maintenance cost — prefer diverse, labeled, few dozen over 5k unlabeled.
• Automatic metrics vs human review — automate gates; sample humans.
• Strict gates vs ship speed — start with parse/citation validity; raise the bar as you learn.

When you are “done enough” to demo externally

• Golden set exists in repo
• CI runs on prompt/model changes
• At least one safety/refuse case
• Known failure modes documented
• Owner for weekly gold updates

Glossary

Micro-project

Write 20 golden cases for one feature with expected citations or tool outcomes. Define a one-line gate (e.g. “≥85% citation-valid and 0 illegal tools”).

Related guided path

Evals, guardrails, safety eval + guardrails lab. Concepts: Agents, RAG, Prompt engineering. Examples: Evals as release gates.