Core Concepts

Privacy and data for AI apps

PII, retention, tenant isolation, and what not to put in prompts, logs, or vector indexes — the AI-for-Everyone society week applied to builders.

40 min

Builders inherit society-week problems

Ng’s AI and Society covers bias, privacy, and labor. Engineers implement the controls: minimize data, isolate tenants, audit access.

Privacy is not a policy PDF — it is which bytes leave your boundary on every LLM call, embed, and trace export.

Hot paths for leaks

  1. Prompts and tool args sent to a third-party API
  2. Vector indexes that embed raw PII
  3. Trace/log systems storing full prompts forever
  4. Shared caches (Redis) without tenant keys
  5. Training / eval sets copied to laptops
  6. Browser-side keys that expose provider accounts
flowchart TD
  User[User data] --> App[Your app]
  App --> LLM[Model provider]
  App --> Vec[Vector DB]
  App --> Logs[Observability]
  LLM -.->|policy| Retain[Retention & training opt-out]
  Vec --> ACL[Tenant ACL filters]
  Logs --> TTL[TTL + redaction]

Practical controls

Control Where
Minimize / redact PII Before embed, log, or vendor call
Per-tenant collections or mandatory metadata ACL Vector DB + search
Separate debug retain vs analytics aggregate Observability
VPC / private endpoints Sensitive workloads
Document provider training opt-out Legal + eng checklist
Encrypt at rest + key hierarchy Indexes, object storage
DLP at gateway Strip secrets / card numbers

Ship rule: default to redact-on-write for traces. Debug with explicit elevate + short TTL.

Tenancy and RAG

Multi-tenant SaaS + RAG is a classic footgun: retrieve neighbor tenant’s chunks → leak. Mitigations:

  • Collection-per-tenant or strict metadata filter on every query
  • Integration tests that attempt cross-tenant retrieval and must fail
  • No shared “global” cache of embeddings across tenants without keys

See Data track chunking/metadata and RAG building blocks.

Prompts, agents, and untrusted data

Tool results and uploaded docs can carry prompt injection and PII. Treat them as data:

  • Delimit untrusted blocks
  • Do not elevate tool text to system policy
  • Scrub secrets from observations before logging
  • HITL for actions that export data externally

Pair with Agents and ReAct and Guardrails.

Bias and safety (builder view)

Privacy ≠ safety. Also measure:

  • Subgroup failure rates on eval sets
  • Refusal behavior on disallowed requests
  • Over-refusal that breaks legitimate workflows

Privacy is necessary but not sufficient for safe products.

Failure modes

Failure Example Fix
Forever traces Full SSN in LangSmith forever Redact + TTL
Index PII Embed raw HR docs Tokenize / minimize fields
Cross-tenant RAG Missing ACL filter Mandatory filter + tests
Shadow IT Employees paste CRM into ChatGPT Approved gateway + training
Eval sprawl Gold sets with real customers Synthetic / consented / scrubbed

Tradeoffs

  • Max logging — great debugging; poor privacy.
  • Heavy redaction — safer; harder incident response.
  • On-prem / VPC models — stronger data control; ops cost (Open-weight vs APIs).

Glossary

Term Meaning
PII Personally identifiable information
TTL Time-to-live for retained data
ACL Access-control list / authorization filter
Data minimization Collect/keep only what the job needs
Training opt-out Provider must not train on your API data

Micro-project

Draft a data-flow diagram marking PII stores, logs, and third-party model calls. Annotate retention for each hop.

Evals, guardrails, safety; RAG ACL metadata on chunks; industry labs on multi-tenant SaaS.

Project checklist0/3 done