AI & Legal Technology Laboratory · Interactive demo

Teaching an MLP to draw sin(x) + cos(2x)

1 input→8 ReLU→8 ReLU→1 linear

A multi-layer perceptron learns a noisy one-variable function. It receives a single input, x (scaled to [−1, 1]), passes it through two hidden layers of 8 ReLU neurons, and outputs a single number ŷ. Press Train to watch gradient descent bend the output curve toward the data. Drag across the fit plot to choose which x flows through the network diagram.

The network

Hover a neuron for its arithmetic

positive weight negative weight neuron firing ReLU output 0

1 · Inputa₀ = x/π

2 · Hidden 1h₁ = ReLU(W₁a₀ + b₁)

3 · Hidden 2h₂ = ReLU(W₂h₁ + b₂)

4 · Outputŷ = w₃·h₂ + b₃

The fit

x ∈ [−π, π]

noisy samples true f(x) network ŷ(x) earlier epochs

Probe input x0.00

Show each layer-2 neuron's contribution to ŷ (they sum, plus b₃, to the orange curve)

Loss

Mean squared error, log scale

training MSE (vs noisy samples) MSE vs true f(x) noise floor σ²

Training

Adam · mini-batch 16

Epoch

Train MSE

–

True MSE

–

Dead ReLUs

–

Learning rate

Noise σ0.20

Samples80

Epochs per frame4