REINFORCE Gradient

Problem

For a Bernoulli policy with probability p = sigmoid(theta), the score of action a in {0, 1} is a - p.

Given the policy parameter theta, the action, the reward, and a baseline, return the REINFORCE gradient estimate (reward - baseline) * (a - p).

Hint

Subtracting a baseline does not bias the gradient but can reduce its variance.

case 1

in: [0,1,1,0]

expected: 0.5

case 2

in: [0,0,1,0]

expected: -0.5

case 3

in: [2,0,2,1]

expected: -0.8807970779778823

case 4

in: [-1,1,0,0.5]

expected: -0.36552928931500245

Starter codePython

def reinforce_gradient(theta, action, reward, baseline):
    # Your code here
    pass

4 test cases · run your code against them in the editor