Q-Value From Transitions

Problem

Compute the action value Q(s,a) from a transition list and a value function.

transitions is a list of [probability, next_state] pairs, reward is the immediate reward for the action, and V holds the state values. Return sum of p * (reward + gamma * V[next]).

Hint

Weight the reward plus discounted next value by each transition probability.

Test cases

case 1

in: [[[0.7,0],[0.3,1]],2,0.9,[1,4]]

expected: 3.71

case 2

in: [[[1,1]],-1,0.5,[0,3]]

expected: 0.5

case 3

in: [[[0.25,0],[0.25,1],[0.5,1]],0,1,[2,4]]

expected: 3.5

case 4

in: [[[1,0]],5,0,[100]]

expected: 5

Starter codePython

def q_value_from_transitions(transitions, reward, gamma, V):
    # Your code here
    pass

4 test cases · run your code against them in the editor