AdaGrad Accumulator Step

Problem

AdaGrad accumulates squared gradients G_new = G + g^2 and takes the step x_new = x - lr * g / sqrt(G_new + eps).

Given x, g, the accumulator G, lr, and eps, return [x_new, G_new] for scalar parameters.

Hint

Update the accumulator first, then scale the step by its square root.

case 1

in: [1,0.5,0,0.1,1e-8]

expected: [0.900000002,0.25]

case 2

in: [0,-2,3,0.5,1e-8]

expected: [0.3779644727392526,7]

case 3

in: [2,1,1,1,0]

expected: [1.2928932188134525,2]

Starter codePython

def adagrad_accumulator_step(x, g, accumulator, lr, eps):
    # Your code here
    pass

3 test cases · run your code against them in the editor