Optimization problems
GD, momentum, Adam, schedules. How models actually learn.
375 problems135 Easy165 Medium75 Hard
50 problems
Gradient Descent
op-001
Mini-Batch SGD for Linear Regression
op-002
Momentum Optimizer
op-003
Adam Optimizer
op-004
Exponential LR Schedule
op-005
Seeded Mini-Batch Indices
op-006
Nesterov Accelerated Descent
op-007
Adagrad Update
op-008
RMSProp Update
op-009
Generic Adam Update
op-010
AdamW Decoupled Weight Decay
op-011
AMSGrad Update
op-012
Lion Sign Update
op-013
Step-Decay Learning Rate
op-014
Cosine Annealing Learning Rate
op-015
Linear Warmup Learning Rate
op-016
One-Cycle Learning Rate
op-017
Inverse-Square-Root Learning Rate
op-018
Polynomial Decay Learning Rate
op-019
Reduce On Plateau Check
op-020
Gradient Clipping By Value
op-021
Gradient Clipping By Norm
op-022
Gradient Centralization
op-023
L2 Weight Decay Step
op-024
L1 Proximal Soft-Threshold
op-025
Early Stopping Patience Check
op-026
Polyak Averaging Combine
op-027
EMA Weights Update
op-028
SWA Average Of Iterates
op-029
Lookahead Interpolation
op-030
SAM Perturbation And Step
op-031
Gradient Noise Injection
op-032
Relative Parameter Change
op-033
Loss Moving Average
op-034
Divergence Detector
op-035
LR Finder Best Rate
op-036
Momentum Warmup Factor
op-037
Bias Correction First Moment
op-038
Hessian-Vector Product Finite Difference
op-039
Newton Step 1D With Finite Differences
op-040
BFGS One Update (2x2)
op-041
Projected Gradient Box Step
op-042
Subgradient Step For Absolute Value
op-043
Coordinate Descent One Coordinate
op-044
Simulated Annealing Accept Test
op-045
Epsilon-Greedy Pick
op-046
Softmax Exploration Probabilities
op-047
UCB Bandit Selection
op-048
Thompson Sampling Pick
op-049
Cross-Entropy Method Elite Mean
op-050
Showing 50 of 375 problems.