SARSA Update

Problem

Apply one SARSA on-policy update and return the updated Q table.

Q[s][a] += alpha * (r + gamma * Q[s_next][a_next] - Q[s][a]). Return a deep copy so the original table is not modified.

Hint

SARSA bootstraps from the next action actually taken, unlike Q-learning.

case 1

in: [[[1,2],[3,4]],0,0,1,1,1,0.1,0.9]

expected: [[1.3599999999999999,2],[3,4]]

case 2

in: [[[1,2],[3,4]],1,0,-1,0,1,0.5,0.5]

expected: [[1,2],[1.5,4]]

case 3

in: [[[0]],0,0,2,0,0,1,0.9]

expected: [[2]]

Starter codePython

def sarsa_update(Q, s, a, r, s_next, a_next, alpha, gamma):
    # Your code here
    pass

3 test cases · run your code against them in the editor