Fletcher-Reeves CG Step

Problem

Perform one nonlinear conjugate gradient step with the Fletcher-Reeves formula.

Compute beta = (grad dot grad) / (grad_old dot grad_old), set the new direction p = -grad + beta * p_old, and return [x + lr * p, p]. Use beta = 0 when the old gradient is zero.

Hint

Fletcher-Reeves resets the momentum when the previous gradient is zero.

Test cases

case 1

in: [[1,1],[1,0],[-1,0],[1,1],0.1]

expected: [[0.85,1],[-1.5,0]]

case 2

in: [[0,0],[2,-2],[1,1],[1,1],0.5]

expected: [[1,3],[2,6]]

case 3

in: [[1,2],[0,0],[0,0],[0,0],1]

expected: [[1,2],[0,0]]

Starter codePython

def fletcher_reeves_step(x, grad, p_old, grad_old, lr):
    # Returns [x_new, p_new]
    # Your code here
    pass

3 test cases · run your code against them in the editor