Author skills

Authoring is product design for agents

A skill is not documentation written for humans and accidentally read by a model. It is operational prose: tight enough to fit a token budget, explicit enough to change behavior, and structured so the host can inject it without surprises. Authoring skills is therefore product design — you are deciding what the agent should reliably do in a domain, what it must never do, and how it recovers when tools fail.

Strong skills share DNA with good runbooks and good prompt libraries, but they differ in lifecycle. Runbooks live in Notion; prompt strings live in code; skills live in version-controlled files with metadata the host understands. That separation lets PMs and domain experts propose edits via PR while engineers keep the loader and validation logic stable.

Before you write a single paragraph, answer three questions:

1. Trigger — Under what user intents should this skill load? ("User asks about refunds," not "billing stuff.")
2. Outcome — What does success look like in one sentence? ("Issue or deny refund with cited policy section.")
3. Stop — When should the agent refuse, escalate, or ask a clarifying question?

If you cannot answer all three, the skill scope is too fuzzy. Split it.

Anatomy of a production-grade skill

Use a consistent template so reviewers know where to look. A proven structure:

# Skill: Refund triage

## When to use
One paragraph: triggers, anti-triggers (when NOT to load this skill).

## Procedure
Numbered steps the agent should follow. Imperative voice.

## Examples
### Good
User: … → Agent: … (with tool calls annotated)

### Bad
Common failure + why it is wrong.

## Failure notes
What breaks in production: stale policy dates, ambiguous order IDs, tool timeouts.

## References
Internal links, policy doc version, owner team.

The When to use section doubles as the embedding/search blurb if you later add semantic skill routing. Keep it under 200 tokens.

Procedure steps should reference tools by exact name (lookup_order, not "check the order system"). If a step is conditional, use explicit if/then bullets rather than narrative paragraphs — models follow structured conditionals more reliably.

Callout — Examples beat adjectives: Writing "be empathetic" is weak. Showing a good refusal message with specific phrasing and a cited policy clause is strong. Aim for two good and one bad example minimum per skill.

Example-driven calibration

Examples are few-shot demonstrations without burning context on every request. Store them in the skill file or a sibling examples/ directory referenced by the skill.

Good examples specify:

• User message (realistic, including typos or missing info)
• Expected tool sequence (if any)
• Final user-facing response shape

Bad examples are equally valuable — they document known confusions: users who ask for refunds on digital goods, agents that hallucinate policy exceptions, tool calls with wrong ID formats.

When labeling bad examples, state the failure class: policy violation, wrong tool, premature answer, missing clarification. Those labels become eval tags later.

Failure notes: design for the second week

Demo skills describe the happy path. Production skills describe what breaks:

• Stale knowledge — "Policy effective_date must be checked against order.created_at; skill owner: #billing-policy."
• Tool ambiguity — "If lookup_order returns multiple matches, ask user to pick; never guess."
• Latency — "If refund API exceeds 5s, return hold message and create async ticket."

Failure notes are where senior engineers encode scar tissue. They reduce repeat incidents more than adding another 500 tokens of general instruction.

Loading skills from disk in the host

Your host agent needs a thin skill registry:

1. Scan skills/**/SKILL.md (or your agreed path).
2. Parse frontmatter: name, description, version, optional tags.
3. Expose descriptions to the router or system prompt as a catalog.
4. On selection, read full body into a delimited block:

<skill name="refund-triage" version="1.2.0">
…full markdown…
</skill>

Log skill_name, version, and token_count on every load. Without logs you cannot debug "the agent forgot policy X" reports.

Validate on load: required sections present, description length under cap, no secrets in plaintext. Fail CI if validation fails — skills are code.

Picking your first two skills

Choose skills that stress different shapes:

• One procedure-heavy skill (multi-step, branching, multiple tools).
• One style/rubric skill (code review, writing tone, classification criteria).

Avoid two skills that overlap triggers — routing confusion shows up immediately in evals.

Suggested pairs for course portfolios:

• incident-triage + commit-message-rubric
• sql-explainer + api-error-response-format
• customer-refund + internal-doc-summarizer

Each must include ≥2 good examples, ≥1 bad example, and ≥3 failure notes.

Review checklist before merge

Treat skill PRs like API PRs:

• Description uniquely identifies trigger vs. neighboring skills
• Procedure steps map to real tools in the host
• Examples tested against current model (behavior drifts across model versions)
• Token count measured; fits budget with headroom for user message
• No PII or secrets in examples (use synthetic IDs)

Callout — Version bumps: Patch version for typo/clarity. Minor for new examples or steps. Major when procedure changes outcomes (refund thresholds, safety rules). Log version in agent traces.

Engineering problem (staff framing)

Author skills with contracts: inputs, side effects, examples, evals.

Diagram — Skill package

flowchart LR
  MD[SKILL.md] --> Res[Resources]
  MD --> Ex[Examples]
  Ex --> Ev[Skill eval]

Precise definitions & mental model

Skill manifests, progressive disclosure, test dialogues.

Tradeoffs — when to use what

Generic skills vs vertical-deep skills.

Failure modes (interview + on-call)

Undocumented side effects; no examples; bitrot.

Production & OSS practices

CI that runs skill evals on change.

Micro-project: Ship 2 skills

In your portfolio under the skills module folder:

1. Implement the disk loader from lesson 8.1's format spec.
2. Author two skills with full templates (procedure, examples, failure notes).
3. Wire the host to list skill descriptions at startup and inject one skill on a test prompt.
4. Capture a trace showing skill name, version, and token count.
5. Document in the module README: what you would promote to a third skill next, and what stays a tool or prompt.

Acceptance: running load_skills() returns two entries; a scripted user message triggers the correct skill load in logs.

Checklist

• Two skills committed with examples and failure notes
• Host loads skills from disk without hard-coded paths
• Trace log includes skill name, version, token count
• CI or local validator rejects malformed skill files
• Module README lists skill inventory and routing rules