Huber Gradient Step

Problem

Perform one gradient descent step for the Huber loss with a single weight w. With residuals r_i = w*x_i - y_i, the gradient is mean(x_i * r_i) when |r_i| <= delta and mean(x_i * delta * sign(r_i)) otherwise.

Return w - lr * gradient; empty X returns w.

Hint

Inside delta the gradient is linear in the residual; outside it is clipped to +/- delta.

Test cases

case 1

in: [[1,2,3],[2,4,6],2,1,0.1]

expected: 2

case 2

in: [[1],[3],0,1,0.5]

expected: 0.5

case 3

in: [[1,2],[0,0],1,1,0.1]

expected: 0.85

case 4

in: [[],[],3,1,0.1]

expected: 3

Starter codePython

def huber_gradient_step(X, y, w, delta, lr):
    # Your code here
    pass

4 test cases · run your code against them in the editor