Emphatic TD Lite

Problem

Update the follow-on trace and emphasis used by emphatic TD.

F_new = gamma * follow_on + interest and M = lam * interest + (1 - lam) * F_new. Return [F_new, M].

Hint

The emphatic weight re-weights updates so they do not vanish under off-policy sampling.

case 1

in: [0,1,0.9,0.5]

expected: [1,1]

case 2

in: [2,3,0.5,0.1]

expected: [4,3.9000000000000004]

case 3

in: [1,1,1,1]

expected: [2,1]

Starter codePython

def emphatic_td_lite(follow_on, interest, gamma, lam):
    # Your code here
    pass

3 test cases · run your code against them in the editor