Citations and failures

Grounding fails quietly

RAG systems look authoritative while inventing details — the model smooths missing evidence into fluent lies. Citations tie each claim to retrieved chunk ids so users and eval scripts can verify support. Failure drills inject bad documents (outdated, contradictory, adversarial) and verify the system refuses or flags conflict instead of blending poison into answers.

This lesson closes the domain RAG milestone: a pipeline from ingest through hybrid retrieval (optional rerank) to grounded generation with measurable citation accuracy and poison resistance on a toy corpus.

Callout — cite chunks, not vibes: A citation is valid only if the quoted span supports the claim. Require chunk id + short evidence quote in structured output.

Grounded generation contract

Extend your structured output schema:

{
  "answer": "Refunds are accepted within 30 days of purchase.",
  "citations": [
    {"chunk_id": "billing#chunk2", "quote": "Returns accepted within 30 days"}
  ],
  "confidence": "high"
}

System prompt rules:

• Answer only from provided context blocks labeled [chunk_id].
• If context insufficient, say I don't have enough information with empty citations.
• Every factual sentence must reference ≥1 citation.
• If chunks conflict, state conflict explicitly; do not merge.

Validate in code: every chunk_id in citations must appear in the retrieved set passed to the model.

Citation eval metrics

On labeled QA set:

Citation precision — fraction of cited chunks that actually support the claim (manual or LLM-judge spot check).

Citation recall — fraction of answers that include necessary citations when answer is correct.

Groundedness rate — answers with no unsupported claims vs gold.

Abstention rate — correct "don't know" on unanswerable queries.

Track separately from BLEU-ish text similarity — fluent wrong answers are the enemy.

Poison and conflict scenarios

Construct toy corpus variants:

Use metadata authority=1 vs authority=0 for poison; prompt model to weight higher authority — but verify with eval, not prompt hope alone.

Optional pre-generation check: if top chunks disagree on numeric field (regex extract days), route to clarification template.

Prompting for grounded answers

Context formatting affects hallucination rate:

Use ONLY the context below. Each block is labeled with an id in square brackets.

[billing#chunk2]
Returns accepted within 30 days of purchase with receipt.

Answer the user. Cite chunk ids for every claim.

Put highest-authority chunks first when prompt budget forces truncation — models overweight early context (lost-in-the-middle is real; rerank order matters).

Separate generation prompt version from retrieval config version in logs — grounding bugs often come from prompt changes, not retrieval.

User-facing failure UX

When abstaining or detecting conflict:

• Say what is missing ("I couldn't find refund timing in your documents").
• List sources consulted even if insufficient.
• Offer next step ("upload policy PDF dated 2024" or "contact support").

Poison detection in toy setup trains you for production monitoring: alert when citation rate drops week-over-week or when single low-authority doc dominates citations.

Automated citation checks

Before returning API response, validate programmatically:

def validate_citations(answer, citations, retrieved_ids):
    for c in citations:
        assert c.chunk_id in retrieved_ids
        chunk_text = chunks[c.chunk_id]
        assert fuzzy_contains(chunk_text, c.quote), "quote not in chunk"

fuzzy_contains allows minor whitespace drift — exact substring too brittle. Failed validation triggers retry with stricter prompt or abstain.

Track citation valid rate alongside answer accuracy in milestone results.json.

Milestone integration

Grounded RAG is the capstone of the retrieval module — wire your best hybrid index, optional reranker, and citation prompt into one answer.py entrypoint. Future modules (agentic RAG, workflows) will call this function as a tool; clean interface now saves refactor pain.

Export a GROUNDED_EVAL.md summary table for your portfolio root README linking to m5 results.

End-to-end RAG checklist

Your m5/ milestone folder should include:

m5/
  ingest/           # chunk + embed + vector db
  retrieve/         # hybrid (+ optional rerank)
  generate/         # prompt with context blocks + schema
  eval/
    qa.jsonl
    results.json
    poison_cases.jsonl
  README.md         # architecture diagram, metrics, failures

Run script: query → retrieve top-k → format context → LLM → validate JSON → return.

Log retrieval ids, prompt version, and full response for every eval row.

Callout — milestone honesty: Publish metrics including failures. "82% grounded, poison case 3/5 passed" beats a demo-only happy path.

UI and API surfacing

Even CLI projects should print citations:

Answer: Refunds within 30 days.
Sources:
  [billing#chunk2] "Returns accepted within 30 days..."

API consumers need stable chunk ids linking to stored text for hover previews.

Engineering problem (staff framing)

Uncited answers are unverifiable. Citation UX + abstain policy are product requirements.

Diagram — Grounded answer path

flowchart TD
  Ctx[Retrieved chunks] --> Gen[Generate]
  Gen --> Cite[Inline citations]
  Gen --> Abs{Supported?}
  Abs -->|no| Abstain
  Abs -->|yes| Answer

Precise definitions & mental model

Grounding, attribution, abstain/clarify, faithfulness vs relevance.

Tradeoffs — when to use what

Force-cite (safer, awkward) vs soft-cite.

Failure modes (interview + on-call)

Invented citations; citing irrelevant chunk; no "I don't know".

Production & OSS practices

Faithfulness evals; UI for open-source; block answers below support threshold.

Micro-project: Poison-doc detection

In m5/grounded_rag/:

1. Extend RAG pipeline with citation schema and validation.
2. Add ≥3 poison or conflict fixtures to eval set.
3. Implement detection or explicit conflict response — document strategy in POISON.md.
4. Run full eval; commit results.json with groundedness and poison pass rate.
5. Update milestone README summarizing architecture and metrics.

Checklist

• Citations reference retrieved chunk ids only
• Abstention works on unanswerable queries
• Poison/conflict cases tested with outcomes logged
• Milestone 5 README complete with repro commands