Descending toward reality

A model never sees the truth directly. It sees observations, scores its mismatch with a loss function, and steps downhill along the gradient. Train it, then shift reality and watch it re-track.

Observations and the model's guess

ObservationsModelHidden reality

Loss landscape: slope × intercept

Darker = lower loss× = true parametersDescent path

Loss (mean squared error)

–

Step

Model

–

Reality

–

Learning rate0.15

The lossAverage squared vertical gap between each observation and the model's line. One number for "how wrong am I right now".

The gradientThe direction in which loss rises fastest. Each step moves the parameters a little the opposite way, scaled by the learning rate.

TrackingWhen reality shifts, the old fit suddenly has high loss, and the same downhill rule pulls the model toward the new truth.