Natural Actor-Critic Step

Problem

Take a natural policy gradient step using the Fisher information matrix.

Invert the 2x2 Fisher matrix, map the gradient through it, and return params + lr * F^{-1} g.

Hint

The natural gradient is invariant to reparameterization of the policy.

case 1

in: [[0,0],[[1,0],[0,1]],[2,3],0.1]

expected: [0.2,0.30000000000000004]

case 2

in: [[0.5,-0.5],[[2,0],[0,4]],[2,4],0.5]

expected: [1,0]

case 3

in: [[1,1],[[2,1],[1,2]],[1,0],1]

expected: [1.6666666666666665,0.6666666666666667]

Starter codePython

def natural_actor_critic_step(params, fisher, grad, lr):
    # Your code here
    pass

3 test cases · run your code against them in the editor