MDP Transition Matrix Build

Problem

Build the policy transition matrix from dense MDP transition lists.

transitions[s][a] holds [probability, next_state] outcomes and policy[s] selects one action per state. Return P where P[s][s2] is the probability of moving from s to s2 under the policy.

Hint

Under a deterministic policy each row of P comes from one action's outcome list.

Test cases

case 1

in: [[[[[1,1]],[[0.5,0],[0.5,1]]],[[[1,0]],[[1,1]]]],[0,1]]

expected: [[0,1],[0,1]]

case 2

in: [[[[[1,1]],[[0.5,0],[0.5,1]]],[[[1,0]],[[1,1]]]],[0,0]]

expected: [[0,1],[1,0]]

case 3

in: [[[[[1,1]]],[[[1,2]]],[[[1,2]]]],[0,0,0]]

expected: [[0,1,0],[0,0,1],[0,0,1]]

Starter codePython

def build_transition_matrix(transitions, policy):
    # Your code here
    pass

3 test cases · run your code against them in the editor