Levenberg-Marquardt Scalar Step

Problem

Take one Levenberg-Marquardt step on a scalar parameter.

Compute delta = -grad / (hess + lam) and return [x + delta, delta]. When lam is 0 this reduces to a Newton step; larger lam makes the step behave more like gradient descent.

Hint

lam interpolates between gradient descent and a Newton step.

Test cases

case 1

in: [1,2,4,0]

expected: [0.5,-0.5]

case 2

in: [1,2,4,1]

expected: [0.6,-0.4]

case 3

in: [0,-3,2,0.5]

expected: [1.2,1.2]

case 4

in: [2,0,5,1]

expected: [2,0]

Starter codePython

def lm_step(x, grad, hess, lam):
    # Returns [x_new, delta]
    # Your code here
    pass

4 test cases · run your code against them in the editor