Dyna-Q Planning Update

Problem

Run n Dyna-Q planning updates using a learned model and a seeded random generator.

Each planning step picks a model entry [s, a, r, s_next] uniformly with rng.randrange(len(model)) and applies a Q-learning backup with the current table. Return the updated Q table.

Hint

Planning replays simulated experience from the model with the same backup rule.

Test cases

case 1

in: [[[0,0],[0,0]],[[0,1,1,1],[1,0,2,0]],3,0,0.5,0.9]

expected: [[0,1.175],[1.5,0]]

case 2

in: [[[1,2],[3,4]],[[0,0,1,1]],2,7,0.1,0.5]

expected: [[1.38,2],[3,4]]

case 3

in: [[[0]],[[0,0,5,0]],0,0,0.1,0.9]

expected: [[0]]

Starter codePython

import random
def dyna_q_planning(Q, model, n, seed, alpha, gamma):
    # Your code here
    pass

3 test cases · run your code against them in the editor