Prompt engineering fundamentals

Prompts are interfaces

The DL.AI + OpenAI short course frames prompting as application development, not magic words. Treat system/user messages like API contracts: version them, test them, refuse silent drift.

A prompt change is a code change. If you would not ship an unreviewed SQL migration, do not ship an unreviewed system prompt.

Two working principles

1. Write clear and specific instructions — role, task, format, constraints, edge cases.
2. Give the model time to think — ask for steps, intermediate checks, or tool use before the final answer when reasoning matters.

Use delimiters (""", XML tags, Markdown sections) so instructions and untrusted user content stay separated.

flowchart TD
  Sys[System: policy + format] --> User[User: task]
  User --> Data[Untrusted data block]
  Data --> Model[LLM]
  Model --> Out[Checked output]

Ship rule: anything from users, web pages, or tool results is untrusted. Delimit it; never let it redefine system policy (Agents and ReAct tool observations).

Message roles (chat APIs)

Role	Typical use
system	Stable policy, output contract, safety
user	Task + untrusted content
assistant	Prior model turns / few-shot exemplars
tool	Observations from tools (agents)

Wrong role placement is a common silent quality bug — especially when porting between providers with different chat templates (Tokenization).

Task patterns (the classic four)

Pattern	Example	Often needs
Summarize	Ticket → 3 bullet status	Length + must-include fields
Infer	Review → sentiment + topics	Label set / schema
Transform	Locale rewrite / tone change	Glossary constraints
Expand	Bullet notes → customer email	Brand voice examples

Add extract as a fifth product pattern: messy text → structured object (Structured outputs).

Few-shot vs instructions

Approach	Use when
Clear instructions only	Format is simple; labels are obvious
Few-shot examples	Edge cases are easier shown than described
Dynamic few-shots	Retrieve similar labeled examples per query

Too many demos → overfit and token cost. Prefer 2–5 sharp examples over 20 mediocre ones. Version them next to the prompt.

Iteration loop

flowchart TD
  Draft[Draft prompt] --> Run[Run golden set]
  Run --> Fail[Inspect failures]
  Fail --> Fix[Tighten instructions / examples]
  Fix --> Run
  Run --> Gate{Pass bar?}
  Gate -->|yes| Ship[Ship + version]
  Gate -->|no| Sys[Add retrieval / tools / schema]

Practical iteration tips

• Change one thing per experiment (instruction vs examples vs model).
• Keep a failure taxonomy (format, factuality, tone, refusal).
• Freeze the gold set while iterating; expand gold in a separate PR.
• Record model id + prompt hash in traces.

When prompting is not enough

• Need private facts → RAG or tools (RAG building blocks)
• Need reliable JSON → structured outputs / validators
• Need behavior change on a domain → fine-tune (Fine-tuning LoRA/QLoRA)
• Need multi-step work → agents with evals
• Need lower latency/cost at scale → smaller model / cache / routing

Failure modes

Symptom	Cause	Fix
Ignores format	Weak contract / no validation	Schema + validator
Prompt injection	Untrusted text in same channel	Delimiters + policy; tool sandboxing
Works on 5 demos only	Overfit	Larger gold; holdout
Silent drift	Unversioned prompt in dashboard	Prompt in git + CI evals
Endless prompt tweaks	Wrong lever	Move to RAG/tools/FT

Tradeoffs

• Long system prompts — more policy coverage; more distraction and cost.
• Few-shots — sharper behavior; brittle if demos stale.
• High temperature — creative writing; bad for tools/JSON.

Glossary

Term	Meaning
System prompt	Stable instructions/policy for the model
Few-shot	In-context examples of desired behavior
Delimiter	Markers separating instructions from data
Prompt injection	Untrusted text tries to override instructions
Golden set	Versioned examples used to score prompt changes

Micro-project

Version three prompts for one task; score on a 20-example golden set; pick a winner with a written tradeoff (quality vs tokens).

Related guided path

Talk to models in the real world — Chat APIs and message roles, structured output, prompt versioning. Pair with Evals fundamentals.