Sims
Live simulations you can play, pause, and step through.
3 simulations · everything runs locally
Six gradient-descent variants race down the same elongated valley. Tune the learning rate and watch momentum and adaptive methods pull ahead — or diverge.
Six optimizers start from the same point on an elongated valley. Momentum and Adam escape the flat direction fast, plain SGD crawls, and AdaGrad's accumulated squared gradients shrink its steps until the rate is raised. Drag the slider and stage your own race.
| Optimizer | Steps | Loss | Status |
|---|---|---|---|
| SGD | — | 7.2900 | In progress |
| Momentum | — | 7.2900 | In progress |
| Nesterov | — | 7.2900 | In progress |
| AdaGrad | — | 7.2900 | In progress |
| RMSProp | — | 7.2900 | In progress |
| Adam | — | 7.2900 | In progress |
Check your intuition
1 / 5At lr 0.08, after 20 steps, which optimizer has the lower loss?
A 2 → h → 1 sigmoid network trained with plain SGD on the same points every epoch. Pick a dataset, add hidden units, and watch the boundary bend to separate the classes. High learning rates scramble it; low rates crawl.
epoch 0 · loss 0.7110 · acc 50% · params 25
Epoch 0. Train loss 0.7110, accuracy 50%. Press Run or Step to start training.
Check your intuition
1 / 5With 8 hidden units at lr 0.50, does the XOR net reach 95% train accuracy within 200 epochs?
A fixed weighted graph, ten nodes. Each step finalizes the closest frontier node and relaxes its edges. Watch the ringed frontier nodes compete and the accent shortest-path tree settle in.
| Node | Distance | Parent | State |
|---|---|---|---|
| A | 0 | — | Frontier |
| B | ∞ | — | Unreached |
| C | ∞ | — | Unreached |
| D | ∞ | — | Unreached |
| E | ∞ | — | Unreached |
| F | ∞ | — | Unreached |
| G | ∞ | — | Unreached |
| H | ∞ | — | Unreached |
| I | ∞ | — | Unreached |
| J | ∞ | — | Unreached |
Start at A. Only the start has distance 0 — press Step to finalize it and relax its edges.
Check your intuition
1 / 5After A is finalized, which node is finalized next?