SMDP Discounted Target

Problem

Compute the semi-Markov decision process target for an option that lasted duration steps.

Return total_reward + gamma^duration * max_q_next, discounting the next-state value once per primitive step consumed by the option.

Hint

Long options are discounted by gamma raised to their duration, not to one.

case 1

in: [5,3,0.9,10]

expected: 12.290000000000001

case 2

in: [0,0,0.5,4]

expected: 4

case 3

in: [1,1,1,2]

expected: 3

case 4

in: [2,2,0.5,8]

expected: 4

Starter codePython

def smdp_discounted_target(total_reward, duration, gamma, max_q_next):
    # Your code here
    pass

4 test cases · run your code against them in the editor