Q-Learning Update

Problem

Apply one Q-learning off-policy update and return the updated Q table.

Q[s][a] += alpha * (r + gamma * max_a' Q[s_next][a'] - Q[s][a]). Return a deep copy so the original table is not modified.

Hint

Use the maximum over the next state's actions regardless of the behavior policy.

case 1

in: [[[1,2],[3,4]],0,0,1,1,0.1,0.9]

expected: [[1.3599999999999999,2],[3,4]]

case 2

in: [[[1,2],[3,4]],1,1,0,0,0.5,0.5]

expected: [[1,2],[3,2.5]]

case 3

in: [[[0]],0,0,2,0,0.25,0.9]

expected: [[0.5]]

Starter codePython

def q_learning_update(Q, s, a, r, s_next, alpha, gamma):
    # Your code here
    pass

3 test cases · run your code against them in the editor