Build vs buy

Synthesis: turn experience into judgment

Modules on modeling, agents, evals, production, and industry labs gave you components. This synthesis module asks: what should your organization own vs rent? Build vs buy for model/agent infrastructure is not ideological — it is a decision under constraints: differentiation, talent, time-to-market, compliance, and total cost of ownership (TCO).

A design review forces explicit tradeoffs before capstone scope freezes.

Framework for agent infra decisions

Evaluate each layer separately — monolithic "build AI" fails:

Layer	Build signals	Buy signals
Foundation model	Regulated domain, unique data moat, SLM suffices	Need frontier reasoning now
Embeddings/RAG	Standard docs, vendor lock-in fear low	Managed vector + parsers fast
Agent orchestration	Novel workflows, deep harness needs	Commodity chat + tools
Evals/guardrails	Domain-specific policy, custom judges	Turnkey if shallow use case
Serving/gateway	High traffic, cost optimization	Low traffic API OK
Observability	Already have OTel stack	SaaS LLM ops acceptable

Differentiation test: If a vendor ships the same feature to all competitors, does your product still win? If yes, buy. If the workflow is the product, build the harness and own evals.

Cost beyond API bills

TCO includes:

• Engineer salaries on platform team
• Incident toil without runbooks
• Fine-tune/serving GPU amortization
• Compliance audit surface (subprocessors, data residency)
• Opportunity cost — capstone time not spent on user features

Quantify roughly — order-of-magnitude beats false precision. $0.002/1k tokens hides when agents loop 40 turns.

Callout — Talent is the hidden constraint: Buying OpenAI is easy; hiring someone who debugs agent trajectories is hard. Record talent assumptions in your decision.

Design review agenda

Run 45-minute review (solo or pair) documented in design_review.md:

1. Context — product, users, compliance tier
2. Requirements — latency, cost cap, offline/VPC, languages
3. Options — at least 2 viable architectures (e.g. all-API vs gateway+SLM)
4. Tradeoff matrix — score 1–5 on cost, speed, control, risk
5. Decision — chosen path + revisit date (e.g. 6 months or 10× traffic)
6. Kill criteria — when to switch (API bill >$X, judge ASR >Y)

Cite ≥2 patterns from industry labs (gateway, harness, authz) as evidence you know what build entails.

Common failure modes

• Resume-driven build — Kubernetes for 10 req/day
• Vendor hug — no eval portability when switching models
• Frozen decision — no revisit when usage 100×

Stakeholder alignment document

Design review appendix: one-page summary for PM — "we buy embeddings, build harness" in plain language. Reduces mid-capstone rewrites when PM assumed full custom stack.

Revisit triggers in practice

Schedule revisit in calendar with concrete triggers: API spend >$500/mo, p95 latency SLO miss 2 weeks running, new VPC requirement from customer contract. Empty revisit dates become never.

Vendor exit strategy

For each bought component, note migration cost: vector DB export format, prompt portability, eval golden independence. Build-vs-buy fails if exit is impossible — document exit sketch even when buying.

Putting it together in practice

ShipAI treats this lesson as executable curriculum, not reading alone. Before marking complete, trace one real request through your portfolio stack and label where this lesson's concepts apply — even if the first pass is messy. Document what broke in the module README; that gap list becomes your next sprint.

Compare your implementation against the industry callouts cited earlier without copying their scale. Name one deliberate simplification you kept (mock auth, SQLite not Postgres, single-region deploy) and one simplification you refuse to ship without (no eval gate, no trace on mutating tools, no fail-closed guardrail on exfil cases). That contrast is what interviewers and graders look for.

Callout — Teach back: Explain this lesson's core tradeoff to a peer in five minutes without slides. If you cannot, re-read the failure modes section and add an example from your own run logs.

Common questions and misconceptions

"Is this overkill for a side project?" Side projects can skip pieces; capstones and production cannot skip knowing the pieces exist. You may waive cost accounting in v1 but your architecture diagram should still show where it would attach.

"Should I rewrite from scratch?" Extend what you built in prior modules — graders reward evolution, not parallel unused folders. Link file paths in your checklist.

"Which metric matters most?" The metric tied to user harm or revenue: policy violations, failed refunds, silent wrong answers — not vanity leaderboard scores.

Extension paths after the micro-project

After the micro-project passes smoke check, choose one extension aligned with your capstone pillar: tighten eval coverage, add a chaos or red-team case, or wire observability into SSE streams. Extensions belong in BACKLOG unless scope freeze explicitly includes them — avoids capstone death by optional polish.

Engineering problem (staff framing)

Decide build/buy/partner per layer with TCO and differentiation analysis.

Diagram — Build vs buy

flowchart TD
  Layer[Layer] --> Diff{Differentiates?}
  Diff -->|yes| Build
  Diff -->|no| Buy

Precise definitions & mental model

TCO, switching costs, data gravity, core vs context.

Tradeoffs — when to use what

Speed now vs flexibility later.

Failure modes (interview + on-call)

Building commodity serving; buying core IP blindly.

Production & OSS practices

ADR per major layer; revisit yearly.

Micro-project: Design review

Deliver design_review.md for your capstone domain:

• All agenda sections filled
• Tradeoff matrix with narrative, not only numbers
• Decision + revisit date + kill criteria
• Explicit build vs buy per layer table

Acceptance: another engineer could implement capstone architecture from this doc alone.

Checklist

• design_review.md with options and matrix
• Decision recorded with revisit date
• Kill criteria quantified
• Industry lab patterns cited as evidence
• Module README links to review