Classifiers and language models produce raw scores. Three small functions turn those scores into probabilities you can act on: the sigmoid for two choices, the softmax for many, and a temperature dial that decides how sharply the probabilities favor the top score. Every number on this page is computed live as you move the controls.
Logistic regression treats its score z as the log-odds of the positive class. Undo the log and the odds, and you get the sigmoid:
Each unit of score multiplies the odds by e ≈ 2.72. A score of 0 is a coin flip. The curve is steepest there and flattens toward 0 and 1, so the score can range over all real numbers while p stays strictly between 0 and 1.
With two features the score is z = w₁x₁ + w₂x₂ + b. The decision boundary is the line where z = 0 and p = 50%. The arrow is the weight vector w: it points toward class A, and its length sets how fast p climbs as you move across the line.
Press Train to run gradient descent on this log loss. Drag the arrow to set w yourself, or click the field to add points. On data that separates cleanly, try λ = 0: the boundary stops moving but ‖w‖ keeps growing, and the model becomes ever more confident. The temperature slider rescales the score as z / T, which changes confidence without moving the boundary.
With more than two classes, each class gets its own score (a logit). Softmax exponentiates every logit so all values are positive, then divides by their total so they sum to 1:
Only differences between logits matter. Adding the same number to every logit changes each ez but leaves the probabilities alone. With two classes, softmax reduces to the sigmoid of the score difference, so logistic regression is the two-class case of softmax.
Temperature divides every logit before the softmax:
At T = 1 you get the model's own distribution. Below 1, gaps between logits widen and the top choice takes over; as T → 0 the softmax becomes argmax, which is greedy decoding. Above 1, gaps shrink and the distribution flattens toward uniform. The ranking never changes; only the confidence does.
Language models pick each next word by sampling from this distribution. Below are invented logits a model might give for the next word. Generate a few sentences at different temperatures and compare the counts to the probabilities.
Bars show probability. The black tick shows the share of samples so far.
| Piece | Formula | Relationship |
|---|---|---|
| Sigmoid | σ(z) = 1 / (1 + e−z) | Inverse of the log-odds. Maps any score to a probability between 0 and 1. |
| Logistic regression | p = σ(w·x + b) | A linear score passed through the sigmoid and trained with log loss, which is binary cross-entropy. |
| Softmax | pk = ezk / Σ ezj | Generalizes the sigmoid to K classes. With K = 2, p₁ = σ(z₁ − z₂). |
| Temperature | softmax(z / T) | Rescales logits. Equivalent to scaling the weights by 1/T, so it changes confidence but never the ranking or the decision boundary. |