Synthetic data loops

Synthetic data helps when filtered; harms when echoed

LLMs generate unlimited synthetic examples — QA pairs, tool trajectories, adversarial prompts — cheaply. Used well, they expand eval coverage and fine-tune sets where human labels are scarce. Used poorly, they collapse diversity (model teaching itself biases), inject subtle errors at scale, and inflate eval scores that do not transfer to production.

Engineering discipline: generate → filter → human spot-check → version → measure lift.

Generate phase

Prompt templates with structured output:

{"user": "...", "expected_tool": "lookup_order", "difficulty": "hard"}

Vary seeds: persona, locale, edge cases, typos. Generate 200 candidates; expect most to discard.

Log generator model version — synthetic data is not stationary across model updates.

Callout — Synthetic eval ≠ synthetic quality: More cases help only if labels are correct. Wrong synthetic gold poisons CI green.

Filter phase

Automated filters:

• Schema validation
• Dedup (embedding distance < threshold)
• LLM judge "is this realistic and label correct?" — calibrated judge only
• Rule checks (no PII patterns, no contradictions with policy doc)

Human spot-check sample 5–10% of kept rows before merge to golden set.

Target keep rate 10–30% — high discard is healthy.

Measure lift vs human-only baseline

Experiment:

• Set A: 20 human-labeled cases
• Set B: 20 human + 80 filtered synthetic
• Run eval on held-out human test set never seen in synthetic generator prompts

Metrics: pass rate delta on held-out human set. Positive lift → synthetic helped generalization. Zero or negative → collapse or overfit to generator biases.

Document in synthetic/lift_report.md.

Distribution collapse warnings

Signs of collapse:

• Vocabulary n-gram diversity drops in kept set
• Eval passes synthetic suite but fails fresh manual probes
• Generator and eval model same family — hidden correlation

Mitigations: multi-model generation, human adversarial review, periodic fresh human seed refresh.

Train vs eval synthetic separation

Never filter synthetic eval cases using the same model that generates training FT data without holdout discipline. Maintain synthetic_eval/ vs synthetic_train/ pipelines — cross-contamination inflates metrics.

Link flywheel lesson: synthetic augments human signal, does not replace.

Provenance metadata

Each synthetic row tags generator_model, prompt_template_version, filter_pipeline_version — when eval drifts, bisect which stage broke. Same discipline as golden set versioning.

Human diversity in loop

Synthetic generators inherit model demographics bias — include explicit instruction blocks for locale/name diversity; human spot-check for stereotype patterns before merge.

Cost of synthetic generation

Generating 200 rows with frontier model has cost — log spend; compare to human labeler hour cost — justify synthetic in flywheel economics section.

Putting it together in practice

ShipAI treats this lesson as executable curriculum, not reading alone. Before marking complete, trace one real request through your portfolio stack and label where this lesson's concepts apply — even if the first pass is messy. Document what broke in the module README; that gap list becomes your next sprint.

Compare your implementation against the industry callouts cited earlier without copying their scale. Name one deliberate simplification you kept (mock auth, SQLite not Postgres, single-region deploy) and one simplification you refuse to ship without (no eval gate, no trace on mutating tools, no fail-closed guardrail on exfil cases). That contrast is what interviewers and graders look for.

Callout — Teach back: Explain this lesson's core tradeoff to a peer in five minutes without slides. If you cannot, re-read the failure modes section and add an example from your own run logs.

Common questions and misconceptions

"Is this overkill for a side project?" Side projects can skip pieces; capstones and production cannot skip knowing the pieces exist. You may waive cost accounting in v1 but your architecture diagram should still show where it would attach.

"Should I rewrite from scratch?" Extend what you built in prior modules — graders reward evolution, not parallel unused folders. Link file paths in your checklist.

"Which metric matters most?" The metric tied to user harm or revenue: policy violations, failed refunds, silent wrong answers — not vanity leaderboard scores.

Extension paths after the micro-project

After the micro-project passes smoke check, choose one extension aligned with your capstone pillar: tighten eval coverage, add a chaos or red-team case, or wire observability into SSE streams. Extensions belong in BACKLOG unless scope freeze explicitly includes them — avoids capstone death by optional polish.

Engineering problem (staff framing)

Synthetic data scales but collapses diversity if unchecked.

Diagram — Synthetic loop

flowchart TD
  Teacher --> Synth --> Filter --> Train --> Student --> Eval

Precise definitions & mental model

Distillation, filtering, diversity, contamination.

Tradeoffs — when to use what

Scale vs authenticity.

Failure modes (interview + on-call)

Model collapse; leaking eval into synth.

Production & OSS practices

Mix real+synth; track lineage.

Micro-project: Generate + filter eval data

Ship:

1. Generator script + 200 raw synthetic rows.
2. Filter pipeline with documented rules and keep rate.
3. lift_report.md comparing eval on human holdout before/after adding synthetic to golden set.
4. Spot-check log (human reviewed N rows, K rejected).
5. Collapse risk section with monitoring ideas.

Acceptance: honest negative result counts if lift absent — document why.

Checklist

• Generator produces structured raw dataset
• Filter rules documented with keep rate
• Human spot-check sample recorded
• Lift measured on human holdout only
• Collapse risks and mitigations written