Guardrails

Gateway-level vs app-level guardrails

Guardrails enforce policy on inputs, outputs, and tool calls — independent of model compliance. Two deployment shapes matter:

• App-level — Middleware in your FastAPI/agent host before/after LLM calls. Fast to ship; each service repeats logic.
• Gateway-level — Shared AI gateway (Uber-style AI gateway pattern) enforcing authz, rate limits, DLP, and model routing for all agents.

Course projects implement app-level middleware with a note on gateway extraction when agent count >1.

Principle: fail closed on high-risk violations — block and log rather than "helpfully" continuing.

PII middleware

Detect and redact before sending to external model providers or logging:

• Email, phone, SSN-like patterns, credit card PAN, API keys in user paste
• Optional NER model for names/addresses (trade latency for recall)

Pipeline placement:

User input → PII scan → [block | redact | allow] → LLM
LLM output → PII scan → user
Tool results → PII scan → model (often forgotten!)

Redaction style: replace with [REDACTED_EMAIL] preserving structure for debugging.

Log pii_block_event with type counts — never log raw matched values.

Callout — Vendor subprocessors: Redacting before API call may be contractually required; document what still leaves your VPC in provider logs.

Policy middleware

Encode product rules deterministically where possible:

• Refund over $X requires human_approval tool path
• export_user_data only role admin
• Answers about legal/medical include disclaimer template (not substitute for legal review)

Pattern:

def policy_check(ctx, proposed_tool_call):
    if proposed_tool_call.name == "issue_refund" and ctx.amount > LIMIT:
        return Block("REFUND_LIMIT", retryable=False)
    return Allow()

Model cannot override Block — return structured error to orchestrator.

Fail closed vs fail open

Document decisions in guardrails.md.

Logging blocks for review

Security and PM teams need review queues:

{"event": "guardrail_block", "rule": "pii_email", "stage": "input", "run_id": "...", "sample_hash": "..."}

Human reviewers sample blocks to tune false positives. Weekly review reduces user friction without opening holes.

Integration with red-team suite

Re-run ASR from lesson 10.4 after guardrails — expect ASR drop on exfil and tool coercion cases. Publish before/after table in module README.

Layered defense depth

App middleware + gateway DLP + tool server authz — overlapping layers so single miss does not exfiltrate. Document which layer catches which red-team case in matrix after ASR rerun.

False positive handling

Aggressive PII rules block legitimate support emails in user paste. Tune patterns; offer user override flow with warning for internal tools only — never for customer-facing without legal review. Log false positive samples for weekly rule tuning.

Putting it together in practice

ShipAI treats this lesson as executable curriculum, not reading alone. Before marking complete, trace one real request through your portfolio stack and label where this lesson's concepts apply — even if the first pass is messy. Document what broke in the module README; that gap list becomes your next sprint.

Compare your implementation against the industry callouts cited earlier without copying their scale. Name one deliberate simplification you kept (mock auth, SQLite not Postgres, single-region deploy) and one simplification you refuse to ship without (no eval gate, no trace on mutating tools, no fail-closed guardrail on exfil cases). That contrast is what interviewers and graders look for.

Callout — Teach back: Explain this lesson's core tradeoff to a peer in five minutes without slides. If you cannot, re-read the failure modes section and add an example from your own run logs.

Common questions and misconceptions

"Is this overkill for a side project?" Side projects can skip pieces; capstones and production cannot skip knowing the pieces exist. You may waive cost accounting in v1 but your architecture diagram should still show where it would attach.

"Should I rewrite from scratch?" Extend what you built in prior modules — graders reward evolution, not parallel unused folders. Link file paths in your checklist.

"Which metric matters most?" The metric tied to user harm or revenue: policy violations, failed refunds, silent wrong answers — not vanity leaderboard scores.

Extension paths after the micro-project

After the micro-project passes smoke check, choose one extension aligned with your capstone pillar: tighten eval coverage, add a chaos or red-team case, or wire observability into SSE streams. Extensions belong in BACKLOG unless scope freeze explicitly includes them — avoids capstone death by optional polish.

Engineering problem (staff framing)

Guardrails are layered filters/policies around the model — defense in depth.

Diagram — Guardrail layers

flowchart LR
  In --> InG[Input filters]
  InG --> Model
  Model --> OutG[Output filters]
  OutG --> Policy[Policy engine]

Precise definitions & mental model

Allow/deny topics, PII filters, schema guards, tool authz.

Tradeoffs — when to use what

Strict filters ↑false positives; loose ↑risk.

Failure modes (interview + on-call)

Single LLM self-check as only control; no bypass audit.

Production & OSS practices

Independent classifiers when stakes high; metrics on block rate.

Micro-project: PII + policy middleware

Ship:

1. Input/output PII middleware with ≥3 pattern classes.
2. Policy block on ≥1 high-risk tool (refund, export, or delete).
3. Fail closed on scanner error (config flag documented).
4. JSONL block logs + sample reviewer doc.
5. ASR before/after on red-team critical subset.

Acceptance: exfil case blocked with log entry; normal golden cases still pass.

Checklist

• PII middleware on input and output paths
• Policy middleware blocks high-risk tool independent of model
• Fail closed behavior documented
• Block events logged without raw secrets
• Red-team ASR improved or risks explicitly accepted