Weight Decay Equivalence

Problem

Convert an L2 regularization coefficient into an equivalent decoupled weight decay.

Under the SGDW convention the decay is applied outside the learning rate, so return l2_coef * lr. This reproduces the same per-step shrinkage as gradient-based L2 regularization.

Hint

When decay is applied directly, it must absorb the learning rate factor.

Test cases

case 1

in: [0.01,0.1]

expected: 0.001

case 2

in: [0.1,0.01]

expected: 0.001

case 3

in: [0,0.5]

expected: 0

case 4

in: [0.05,1]

expected: 0.05

Starter codePython

def weight_decay_equivalence(l2_coef, lr):
    # Your code here
    pass

4 test cases · run your code against them in the editor