SignSGD Update

Problem

Perform one SignSGD update on a parameter vector.

Return w_i - lr * sign(g_i) elementwise, where sign is 1 for positive gradients, -1 for negative gradients, and 0 for zero gradients. Every coordinate moves by the same amount lr.

Hint

SignSGD discards the gradient magnitude, so only the sign drives each step.

Test cases

case 1

in: [[1,2,3],[0.5,-0.5,0],0.1]

expected: [0.9,2.1,3]

case 2

in: [[0],[-2],0.5]

expected: [0.5]

case 3

in: [[5,-5],[1,1],1]

expected: [4,-6]

Starter codePython

def signsgd_update(w, g, lr):
    # Your code here
    pass

3 test cases · run your code against them in the editor