Adagrad Update

Problem

Perform one Adagrad update on a scalar parameter.

Given x, gradient g, and accumulated squared gradient accum, compute accum_new = accum + g * g and x_new = x - lr * g / (sqrt(accum_new) + eps). Return [x_new, accum_new].

Hint

Add g squared to the accumulator first, then take its square root.

Test cases

case 1

in: [1,0.5,0,0.1,1e-8]

expected: [0.900000002,0.25]

case 2

in: [0,-2,1,0.01,1e-8]

expected: [0.00894427186999916,5]

case 3

in: [2,0,0.5,0.1,1e-8]

expected: [2,0.5]

case 4

in: [-1,3,4,0.05,1e-8]

expected: [-1.0416025146015075,13]

Starter codePython

def adagrad_update(x, g, accum, lr, eps):
    # Returns [x_new, accum_new]
    # Your code here
    pass

4 test cases · run your code against them in the editor