GAE Full Vector

Problem

Compute the full generalized advantage estimation vector.

Walk backwards with delta_t = rewards[t] + gamma * values[t + 1] - values[t] and A_t = delta_t + gamma * lam * A_{t+1}, where the advantage after the last step is 0. Return the list of advantages.

Hint

values has one more entry than rewards; the recursion runs from the episode end.

Test cases

case 1

in: [[1,1],[0,2,3],0.9,0.5]

expected: [3.565,1.7000000000000002]

case 2

in: [[1,1],[0,2,3],0.9,1]

expected: [4.33,1.7000000000000002]

case 3

in: [[1,1],[0,2,3],0.9,0]

expected: [2.8,1.7000000000000002]

case 4

in: [[0,1,2],[0,0,0,0],0.9,0.9]

expected: [2.1222000000000003,2.62,2]

Starter codePython

def gae_full(rewards, values, gamma, lam):
    # Your code here
    pass

4 test cases · run your code against them in the editor