Adam Optimizer

Problem

Minimize f(x) = x^2 using the Adam optimizer.

Initialize m = 0, v = 0. For t = 1 to n_iters: grad = 2 * x m = beta1 * m + (1 - beta1) * grad v = beta2 * v + (1 - beta2) * grad^2 m_hat = m / (1 - beta1^t) v_hat = v / (1 - beta2^t) x = x - lr * m_hat / (sqrt(v_hat) + eps)

Return the final value of x.

Hint

Bias-correct m and v by dividing by (1 - beta^t). Watch the order: v_hat ** 0.5 + eps.

Test cases

case 1

in: [10,0.1,0.9,0.999,1e-8,500]

expected: 0

case 2

in: [5,0.1,0.9,0.999,1e-8,1000]

expected: 0

case 3

in: [0,0.1,0.9,0.999,1e-8,100]

expected: 0

Starter codePython

def adam_gradient_descent(start, lr, beta1, beta2, eps, n_iters):
    # Minimize f(x) = x ** 2
    # Your code here
    pass

3 test cases · run your code against them in the editor