Library
Real-world examples
Blog-length deep dives into how production AI systems are shaped — serving gateways, RAG, agents, cost, and evals. Patterns inspired by public industry knowledge, not invented insider quotes. Each example maps onto browse tracks and guided modules — see the curriculum outline.
Evals as release gates: shipping LLM changes without vibes
Treat prompt and model changes like code — golden sets, graded rubrics, and CI gates that block regressions before customers notice.
Open-source stack for an AI feature: vLLM, Ollama, and graph orchestrators
A pragmatic OSS topology for teams that want ownership — local iteration, GPU serving, orchestration graphs, and evals — without boiling the ocean.
Multi-tenant AI SaaS: isolation, quotas, and noisy neighbors
Tenant A’s prompt cache must not serve tenant B. Design isolation for data, models, rate limits, and spend — before you scale seats.
Fine-tune vs prompt vs RAG: a decision framework
Choose the cheapest lever that hits quality — with clear criteria, anti-patterns, and a path to escalate only when evidence demands it.
Agents in production: ReAct loops, timeouts, and human-in-the-loop
Agent demos loop forever. Production agents need budgets, idempotent tools, escalation paths, and explicit stop conditions.
Observability for LLM apps: traces, evals, and user feedback
Logs aren’t enough. Wire traces, offline eval suites, and closed-loop feedback so quality regressions are detectable like latency pages.
Cost control for LLM apps: cache, shrink, and route
Unit economics break demos. Practical levers — semantic cache, smaller models, cascades, and prompt budgets — with the failure modes of each.
Marketplace ranking meets LLMs: Uber/Airbnb-style re-rank patterns
Classical IR and marketplace rankers still own the candidate funnel — LLMs shine as re-rankers and explainers when latency and cost allow.
Building a ChatGPT-like product: streaming, tools, and memory
The product surface looks simple — chat — but production needs streaming contracts, tool sandboxes, memory tiers, and hard latency budgets.
Production RAG: chunking, hybrid search, rerank, and eval gates
A shippable RAG architecture — from document ingestion to grounded answers — with the failure modes teams hit after the demo works.
Netflix-style LLM gateway: batching, KV cache, and one API
How to put a stable OpenAI-compatible gateway in front of heterogeneous models — continuous batching, prefix KV reuse, and routing without rewriting every client.