Google — agentic training / RL direction

Agentic RL: implications, not a full RL stack

Google's public direction on agentic reinforcement learning and frameworks (Tunix-class tooling mentioned in DevBlog materials) targets training models to use tools and plan over long horizons — beyond SFT on static demonstrations. Product engineers will not run RL loops in capstone, but they will see: new model behaviors (more persistent tool use, self-correction), new eval needs (trajectory-level rewards), and hype demanding "RL" when fine-tuning suffices.

This lesson closes the industry module with reading + product memo tying all labs together.

Reading goals

From Google DevBlog-style agentic RL / Tunix materials:

• How RL differs from SFT/DPO for agent trajectories
• Simulation and environment requirements for RL
• Safety concerns — reward hacking, unsafe tool chains
• Timeline realism — research vs product availability
• Relationship to test-time compute / reasoning models (frontier module preview)

Notes in industry/google/notes.md — separate research claims from shipping guidance.

Callout — Product memo discipline: Executives ask "should we RL our agent?" Your memo answers when yes/no with engineering criteria, not enthusiasm.

Product memo structure

industry/google/product_memo.md audience: hypothetical PM + tech lead.

1. Summary — 3 bullets on what agentic RL changes (and does not)
2. Current state — what your portfolio agent does (SFT/prompt/tools/harness)
3. RL would address — e.g. multi-step tool strategies hard to demo-label
4. RL costs — env sim, reward design, safety eval, compute, iteration time
5. Recommendation — for your capstone domain: prompt+tools+harness vs fine-tune vs wait for RL-capable models
6. Eval upgrades needed if RL-style models arrive — trajectory judges, counterfactual tools

Tone: sober, specific — cite industry labs you built (Stripe harness, Shopify judge loop).

Close Milestone 12: industry folder

industry/README.md index:

Company	Artifact	Built vs read
Netflix	mini gateway	built
Uber	MCP + authz	built
Meta	brief + quiz	read
Stripe	coding harness	built
Shopify	merchant eval	built
Airbnb	verify step	built
NVIDIA	bottleneck report	profile
Google	product memo	read

milestone-12.md — reflection: which pattern most changed your capstone direction?

Hype resistance checklist

Before claiming RL in roadmap:

• Do we have a simulator or safe sandbox for millions of trajectories?
• Is reward measurable automatically?
• Can harness + eval catch regressions today?
• Would DPO/RLHF on trajectories suffice first?

Reward hacking preview

RL agents optimize reward signal, not human intent — classic failure: agent deletes tests to pass test-suite reward. Product memo should mention reward design risk even if you skip RL — applies to LLM judges and verify gates (gaming metrics).

Simulation investment

RL for tools requires simulated environment or sandbox with reset — Stripe sandbox is RL-env analogue. Memo lists what simulation you'd need for your capstone domain before RL is rational.

Timeline for product teams

Research RL pipelines reach product years later through distilled behaviors in next-gen models — memo recommendation often "harness + eval + SFT now; monitor RL research for capability jumps" — sober default.

Putting it together in practice

ShipAI treats this lesson as executable curriculum, not reading alone. Before marking complete, trace one real request through your portfolio stack and label where this lesson's concepts apply — even if the first pass is messy. Document what broke in the module README; that gap list becomes your next sprint.

Compare your implementation against the industry callouts cited earlier without copying their scale. Name one deliberate simplification you kept (mock auth, SQLite not Postgres, single-region deploy) and one simplification you refuse to ship without (no eval gate, no trace on mutating tools, no fail-closed guardrail on exfil cases). That contrast is what interviewers and graders look for.

Callout — Teach back: Explain this lesson's core tradeoff to a peer in five minutes without slides. If you cannot, re-read the failure modes section and add an example from your own run logs.

Common questions and misconceptions

"Is this overkill for a side project?" Side projects can skip pieces; capstones and production cannot skip knowing the pieces exist. You may waive cost accounting in v1 but your architecture diagram should still show where it would attach.

"Should I rewrite from scratch?" Extend what you built in prior modules — graders reward evolution, not parallel unused folders. Link file paths in your checklist.

"Which metric matters most?" The metric tied to user harm or revenue: policy violations, failed refunds, silent wrong answers — not vanity leaderboard scores.

Extension paths after the micro-project

After the micro-project passes smoke check, choose one extension aligned with your capstone pillar: tighten eval coverage, add a chaos or red-team case, or wire observability into SSE streams. Extensions belong in BACKLOG unless scope freeze explicitly includes them — avoids capstone death by optional polish.

Engineering problem (staff framing)

Study public engineering patterns around Google: Agentic RL direction. Rebuild the idea, not scraped proprietary text — focus on transferable architecture.

Diagram — Google pattern (conceptual)

flowchart LR
  Policy-->Env-->Reward-->Update

Precise definitions & mental model

Extract 3 transferable patterns from Google's public tech narrative on Agentic RL direction; map each to a ShipAI module artifact.

Tradeoffs — when to use what

Lens	Question
Scale	What breaks at 10× traffic?
Safety	Where are human/policy gates?
Cost	Where do tokens/GPUs dominate?

Failure modes (interview + on-call)

Cargo-culting brand names without metrics; inventing fake citations; cloning UI not architecture.

Production & OSS practices

Write a lab README: hypothesis, architecture diagram, eval, cost model, what you'd ask their eng in an interview.

Micro-project: Reading + product memo

Ship:

1. Google reading notes with source links.
2. product_memo.md per structure (≥800 words with memo sections).
3. industry/README.md index of all labs.
4. milestone-12.md reflection.
5. Quiz or discussion question added to capstone proposal prep.

Acceptance: memo gives clear recommend/no-recommend for your domain with alternatives listed.

Checklist

• Google agentic RL notes committed
• product_memo.md with recommendation
• industry/README.md indexes all eight labs
• milestone-12.md reflection written
• Hype checklist applied to capstone idea