REINFORCE Log-Prob Gradient

Problem

Compute the REINFORCE score-function gradient estimate for a trajectory.

Walk backwards to build the return-to-go G_t = rewards[t] + gamma * G_{t+1}, then sum log_probs[t] * G_t over every timestep. Return the scalar gradient estimate.

Hint

Weight each action's log-probability by the return that followed it, not by the full episode return.

Test cases

case 1

in: [[-0.5,-1],[1,2],0.9]

expected: -3.4

case 2

in: [[-0.2,-0.3,-0.4],[1,1,1],1]

expected: -1.6

case 3

in: [[0,0],[1,2],0.9]

expected: 0

case 4

in: [[-1],[-3],0.5]

expected: 3

Starter codePython

def reinforce_gradient(log_probs, rewards, gamma):
    # Your code here
    pass

4 test cases · run your code against them in the editor