Loss, gradients, and overfitting

The training control loop

Machine learning training is a feedback loop. You have parameters (weights), data (inputs and labels), and a procedure that nudges parameters to make better predictions. Four steps repeat until you stop:

1. Forward pass — inputs flow through the model; you get predictions.
2. Loss computation — a scalar measures how wrong those predictions are.
3. Backward pass (backprop) — gradients tell you how each parameter would change the loss.
4. Optimizer step — parameters update in the direction that reduces loss.

This loop is the same whether you are training a logistic regression, a multilayer perceptron, or a transformer. The loss function and the architecture change; the control structure does not.

predictions = model(inputs)
loss = loss_fn(predictions, labels)
loss.backward()          # compute gradients
optimizer.step()         # update weights
optimizer.zero_grad()    # clear gradients for next step

You will implement this loop concretely in PyTorch later in this module. For now, focus on reading the curves that the loop produces — they tell you whether learning is happening, whether it is generalizing, and when to stop.

Loss functions: what are we minimizing?

The loss function (also called objective or cost) converts model outputs and ground truth into a single number you can minimize.

Cross-entropy deserves extra attention because it appears everywhere in AI engineering — classification, fine-tuning, RLHF reward modeling. Given a probability distribution over classes and a true label, cross-entropy measures how many "nats" of surprise your model assigns to the correct answer. A model that assigns 99% to the wrong class gets a huge loss; a model that assigns 99% to the correct class gets a tiny loss.

For regression, MSE squares the error so outliers dominate. Mean absolute error (MAE) is more robust but has a non-smooth gradient at zero — optimizers handle both in practice.

Callout — loss is not accuracy: A model can have low cross-entropy but mediocre accuracy (well-calibrated but cautious), or high accuracy with weird loss (confident on easy examples only). Always log both during development.

Gradients: which way to move?

A gradient is a vector of partial derivatives: for each parameter, how much would the loss change if you nudged that parameter infinitesimally?

Training moves parameters in the negative gradient direction — steepest descent on the loss surface. In high dimensions (millions of parameters), you never see the full surface; you see local slopes at your current point.

Intuition with a single weight: imagine loss as a valley. The gradient tells you which direction is uphill. You step downhill. With many weights, the same idea applies in millions of dimensions simultaneously.

Learning rate controls step size. Too large: you bounce around or diverge. Too small: training crawls and may stall in bad local minima (though modern nets rarely get stuck in truly bad minima — flat regions are the bigger problem).

Common optimizers:

• SGD — simple, noisy, works with momentum.
• Adam — adaptive per-parameter learning rates; default choice for many projects.
• AdamW — Adam with decoupled weight decay; standard for transformers.

You do not need to derive these yet. You need to recognize that when training misbehaves — loss is NaN, loss flatlines, val loss explodes — the first suspects are learning rate, loss scale, and data bugs.

Overfitting: when the model memorizes instead of generalizes

Overfitting means the model fits training noise rather than the underlying pattern. It performs well on data it has seen and poorly on new data from the same distribution.

The classic signature on train/val curves:

Underfitting also happens when your model is too simple for the task, labels are noisy, or features are weak. Do not assume "add layers" fixes everything — sometimes you need better data, not a bigger net.

Regularization: responses to overfitting

When val loss diverges from train loss, you have several levers:

• Early stopping — halt training when val loss stops improving; simplest and often best.
• Weight decay (L2 regularization) — penalize large weights; encourages simpler functions.
• Dropout — randomly zero activations during training; prevents co-adaptation of neurons.
• More data — the most reliable fix when you can get it.
• Data augmentation — synthetic variation (flips, crops, paraphrases) expands effective training size.
• Smaller model — counterintuitive but effective when you are clearly overfitting a tiny dataset.

Callout — val loss going up is not always overfitting: Distribution shift between train and val splits, a misconfigured val loader, or label leakage can mimic overfitting. Always verify your split before reaching for regularization.

Reading curves like an engineer

When you plot train and val loss per epoch, you are reading a diagnostic instrument — like an EKG for your training run.

Healthy run: Both curves decrease smoothly. Val loss may be noisier than train (fewer val samples per epoch). Gap between them is small and stable.

Overfitting run: Train keeps dropping; val bottoms out then rises. The epoch where val was minimum is your early-stop candidate.

Underfitting run: Both curves plateau high. Try more capacity, lower LR with longer training, or inspect labels.

Bug run: Loss is NaN, negative when it should not be, or identical across epochs. Check data types, loss function choice, and whether gradients are flowing (lesson 2.4 covers this).

Log per-step or per-epoch consistently. Compare runs with the same val set and same metric definition. A val loss of 0.4 in one run and 0.4 in another only means something if both use identical preprocessing and batching.

Engineering problem (staff framing)

Read loss curves; capacity and regularization are product decisions.

Diagram — Fit regimes

flowchart LR
  U[Underfit] --> S[Sweet spot] --> O[Overfit]

Precise definitions & mental model

Loss, gradients, overfit, regularization (decay/dropout/early stop).

Tradeoffs — when to use what

More params ↑capacity↑overfit risk; more data ↓overfit; early stop cheap.

Failure modes (interview + on-call)

Train-only metrics; LR explode/plateau; early stop on test.

Production & OSS practices

Log train/val; NaN alerts; best-by-val checkpoints.

Deep dive (FAANG / OSS bar)

Push «loss-gradients-overfitting» past tutorial depth: write the interface contract (inputs/outputs/invariants), list three measurable metrics, and name two degrade modes if the happy path fails. Add a short threat note: what an attacker or noisy tool result could do, and which layer catches it (schema, policy, HITL, or eval gate).

flowchart LR
  Contract[Interface contract] --> Metrics
  Metrics --> Degrade[Degrade modes]
  Degrade --> Threat[Threat + control]

Micro-project: Train/val curves

In m2/curves/:

1. Train a small model (even a 2-layer MLP on MNIST subset or a synthetic regression task) with intentional capacity to overfit a tiny dataset — e.g. 500 training samples, 100k parameters.
2. Save history.json with per-epoch train loss and val loss arrays.
3. Plot both curves with matplotlib or plotly; commit curves.png.
4. Write 5–10 lines in a short NOTES.md: the epoch you would early-stop, why, and what regularization you would try next if this were a real project.

Your plot should clearly show the divergence between train and val. If both curves look perfect, you may not have enough capacity or your dataset is too easy — increase model size or reduce training data until overfitting is visible. The goal is to see the phenomenon, not to achieve good val loss.

Optional extension: run three configs (no regularization, weight decay, early stop) and overlay curves on one chart.

Checklist

• history.json with per-epoch train and val loss
• curves.png committed with labeled axes
• Early-stop rationale written with specific epoch number
• You can explain overfitting vs underfitting vs bug from the curves alone