API keys, budgets, and responsible use

Keys without leaking them

You will call hosted APIs (OpenAI-compatible endpoints, Anthropic, Google, and others) and, later in the course, local OpenAI-compatible servers (Ollama, vLLM). From day one, treat API keys like production database credentials — because that is what they are. A leaked key can burn your budget in hours and exfiltrate data through your account.

The .env pattern

Secrets live in .env. Templates live in .env.example with empty placeholders. Code loads via python-dotenv or your framework's equivalent. Git ignores .env permanently.

# .env.example — commit this
OPENAI_API_KEY=
OPENAI_BASE_URL=https://api.openai.com/v1
# Optional: default model for course drills (use a cheap one)
SHIPAI_DEFAULT_MODEL=gpt-4.1-mini

# .env — NEVER commit this
OPENAI_API_KEY=sk-proj-...
OPENAI_BASE_URL=https://api.openai.com/v1
SHIPAI_DEFAULT_MODEL=gpt-4.1-mini

Your .gitignore must include .env. Before every push, verify with git status that no secret file is staged. If you accidentally commit a key, rotate it immediately at the provider — deleting the commit from history is not enough once pushed.

Provider hygiene

• Prefer project-scoped keys with spend limits when the provider allows (OpenAI, Anthropic, and others offer budget caps).
• Use separate keys for course work vs. employer work vs. side projects.
• Never paste keys into lesson notes, Slack, screenshots, or LLM chat windows — chat logs are not secret stores.
• For team or portfolio demos, use environment variables in CI, not hardcoded fallbacks.

Callout — OpenAI-compatible is a protocol: Many providers and local servers expose the same /v1/chat/completions shape. ShipAI code often works across vendors by changing OPENAI_BASE_URL and model name — but pricing and rate limits differ per provider.

Budgets are part of the craft

AI engineering without cost awareness is demo engineering. A prototype that loops an agent ten times per user message may cost cents in development and dollars per active user in production. The deployment and cost module goes deep; the habit starts here.

Set a monthly cap

Pick a number you can afford to burn on learning — even $10–20 is enough if you default to small models for drills. Write it in m0/README.md:

## Personal API budget
- Monthly cap: $15
- Default drill model: gpt-4.1-mini (or equivalent cheap tier)
- Kill switch: delete/disable key if daily spend > $3

Model tier discipline

Not every call needs the frontier model. Use a tiered strategy:

Escalate model size only when eval scores justify the cost — a pattern you will formalize in the evals module.

Log every call early

When agents arrive in later modules, token usage explodes: system prompts, tool results, multi-turn history, reflection loops. If you only start logging at that point, you will not know what "normal" looks like. The micro-project below establishes JSONL logging from the first call.

Rough cost estimation

Providers bill by tokens. For learning, approximate costs are fine:

est_usd ≈ (prompt_tokens × price_per_1M_input + completion_tokens × price_per_1M_output) / 1_000_000

Hardcode a price table in a comment at the top of your script and refresh it when you notice drift. Exact accounting matters for production; directional accuracy matters for learning.

Responsible use (short, non-lawyer)

This is not legal advice — it is engineering hygiene.

Employer and client data. Do not send proprietary employer data, customer PII, or unreleased product details to third-party APIs without explicit policy clearance. Many companies have approved vendors and data-handling rules. When in doubt, use synthetic data or public datasets until you understand the constraints.

Abuse and ToS. Do not use course projects to harass, scrape illegally, bypass authentication, generate spam, or evade content policies. Providers monitor abuse patterns; your key can be revoked without refund.

Training data and licenses. Fine-tuning on random web data without understanding licenses creates legal and quality risk. The SLM module addresses dataset choices; until then, prefer public or synthetic datasets with clear terms.

Local vs. cloud for sensitive prompts. If prompts cannot leave your VPC, closed APIs are off the table unless your org has a private deployment contract. That constraint drives the open-weight vs. API decision you will analyze in the field-map module.

Engineering problem (staff framing)

LLM bugs become invoices. Control secrets, budgets, and abuse before tool-calling agents.

Diagram — Spend control plane

flowchart LR
  Env[.env / vault] --> App[Scripts]
  App --> API[Provider]
  API --> Bill[Usage]
  Bill --> Alert[Budget alert]
  Alert --> Kill[Circuit breaker]

Precise definitions & mental model

Key=credential; budgets by feature; least-privilege keys; responsible data handling in prompts.

Tradeoffs — when to use what

Failure modes (interview + on-call)

Infinite agent loops; logging PII/keys; sharing prod keys in chats.

Production & OSS practices

Vault secrets, burn-rate alerts, COST.md for eval suites, rotate on leak.

Deep dive (FAANG / OSS bar)

Push «api-keys-budgets-responsible-use» past tutorial depth: write the interface contract (inputs/outputs/invariants), list three measurable metrics, and name two degrade modes if the happy path fails. Add a short threat note: what an attacker or noisy tool result could do, and which layer catches it (schema, policy, HITL, or eval gate).

flowchart LR
  Contract[Interface contract] --> Metrics
  Metrics --> Degrade[Degrade modes]
  Degrade --> Threat[Threat + control]

Micro-project: First logged LLM call

Write m0/first_call/main.py that:

1. Loads OPENAI_API_KEY from environment (via python-dotenv). Fail fast with a clear error if missing.
2. Sends one chat completion request: prompt "Reply with exactly: shipai-ready" (use your configured default model).
3. Verifies the response content matches (or note mismatch in the log — debugging wrong outputs is part of the job).
4. Appends one JSON line to m0/first_call/calls.jsonl with: timestamp (ISO 8601), model name, prompt tokens, completion tokens, estimated USD cost.

Example log line:

{"ts":"2026-08-11T14:32:01Z","model":"gpt-4.1-mini","prompt_tokens":12,"completion_tokens":4,"est_usd":0.0001}

Use httpx or the official OpenAI Python SDK — either is fine. Pin the SDK version in your lockfile when you add it.

Add m0/first_call/README.md with: how to run, which env vars are required, and your monthly budget statement (copy from m0/README.md or link to it).

Checklist

• .env ignored; .env.example committed
• One successful logged call
• Personal monthly budget written in m0/README.md