L2 Weight Decay Step

Problem

Perform one gradient step with L2 weight decay on a scalar weight.

The update is w_new = w - lr * (grad + weight_decay * w). Return the new weight.

Hint

The decay term weight_decay * w is added to the gradient before the step.

case 1

in: [1,0.5,0.1,0.01]

expected: 0.949

case 2

in: [0,-2,0.01,0.1]

expected: 0.02

case 3

in: [2,0,0.5,0.2]

expected: 1.8

Starter codePython

def l2_weight_decay_step(w, grad, lr, weight_decay):
    # Your code here
    pass

3 test cases · run your code against them in the editor