Potential Shaping Return Offset

Problem

Measure how potential-based shaping changes the discounted return.

Compute the plain discounted return of rewards, the return of the shaped rewards r_t + gamma * potentials[t + 1] - potentials[t], and return their difference. potentials has one more entry than rewards and an empty reward list returns 0.0.

Hint

The offset telescopes to gamma^T * phi(s_T) - phi(s_0), independent of the rewards.

Test cases

case 1

in: [[1,2],[0,1,2],0.9]

expected: 1.62

case 2

in: [[1],[1,1],0.9]

expected: -0.10000000000000009

case 3

in: [[2,0],[2,0,0],0.5]

expected: -2

case 4

in: [[],[3],0.9]

expected: 0

case 5

in: [[0,4],[1,3,2],1]

expected: 1

Starter codePython

def shaping_return_offset(rewards, potentials, gamma):
    # Your code here
    pass

5 test cases · run your code against them in the editor