Professor Thinh Nguyen's AI & Legal Technology Laboratory
Networks that remember
A recurrent neural network reads a sequence one step at a time and carries a hidden state from each step to the next. An LSTM adds gates that decide what to keep, what to write, and what to reveal. The model below is a real LSTM training in your browser right now, written from scratch: forward pass, backpropagation through time, and the Adam optimizer.
Training (one step ahead)Forecast (80 steps, no peeking)
Everything left of step 240 is training data. Past that line the network gets no more real values: each forecast is fed back in as the next input, so small mistakes compound.
Switch to Vanilla RNN on Mackey–Glass and compare the forecast error.
Shrink the window below one period of the square wave (32 steps) and watch it lose the rhythm.
Add noise: the training error rises, but the forecast often stays smooth.
Change the window mid-training. The weights are shared across time steps, so nothing needs rebuilding.
Step through time
One cell, unrolled
Each column is the same cell with the same weights, applied at successive time steps. It starts with an empty memory, reads xt, updates its hidden state ht (each row is one hidden unit: orange positive, blue negative), passes ht to the right, and predicts the next value. Tap a column to inspect it, or a row to pick a unit.
Bottom row: how much of an error at the last step reaches back to each earlier step during backpropagation.