REINFORCE Baseline Advantage

Problem

Compute return-to-go values with a constant baseline subtracted (the REINFORCE variance-reduction trick).

Walk backwards to build G_t = rewards[t] + gamma * G_{t+1}, subtract baseline from each, and return the list. An empty episode returns [].

Hint

The baseline does not change the expected gradient but lowers its variance.

case 1

in: [[1,2,3],0.5,0]

expected: [2.75,3.5,3]

case 2

in: [[1,2,3],0.5,3]

expected: [-0.25,0.5,0]

case 3

in: [[-1],0.9,0]

expected: [-1]

case 4

in: [[],0.9,1]

expected: []

Starter codePython

def reinforce_baseline_advantage(rewards, gamma, baseline):
    # Your code here
    pass

4 test cases · run your code against them in the editor