Q(Sigma) Lite

Problem

Compute the Q(sigma) target, which interpolates between expected SARSA and Q-learning.

With E = policy_probs . q_next and G = max(q_next), return r + gamma * (sigma * E + (1 - sigma) * G); a done episode returns just the reward.

Hint

Sigma of 1 is expected SARSA; sigma of 0 is Q-learning.

case 1

in: [1,false,0.9,0.5,[0.5,0.5],[2,4]]

expected: 4.15

case 2

in: [0,false,1,0,[0.25,0.75],[1,5]]

expected: 5

case 3

in: [2,true,0.9,0.5,[0.5,0.5],[1,1]]

expected: 2

Starter codePython

def q_sigma_lite(reward, done, gamma, sigma, policy_probs, q_next):
    # Your code here
    pass

3 test cases · run your code against them in the editor