Value Iteration Grid Step

Problem

Take one value iteration backup on a line of states.

From state i the agent may stay or move to an adjacent state within the line, so V_new[i] = rewards[i] + gamma * max(V[i - 1], V[i], V[i + 1]) over valid indices. Return the updated value list.

Hint

Boundary states have only two candidate successors, including themselves.

Test cases

case 1

in: [[1,2,3],[0,0,0],0.9]

expected: [1.8,2.7,2.7]

case 2

in: [[0,0],[1,2],0.5]

expected: [1,2]

case 3

in: [[5],[1],0.9]

expected: [5.5]

Starter codePython

def value_iteration_step(V, rewards, gamma):
    # Your code here
    pass

3 test cases · run your code against them in the editor