Importance Sampling Ratio

Problem

Compute the importance-sampling ratio of a sequence of actions.

Multiply target_probs[a] / behavior_probs[a] over every action in order. An empty action list gives a ratio of 1.0; assume behavior probabilities are positive.

Hint

The ratio corrects a trajectory sampled by the behavior policy to the target policy.

Test cases

case 1

in: [[0.5,0.5],[0.25,0.75],[0,1]]

expected: 1.3333333333333333

case 2

in: [[0.5,0.5],[0.25,0.75],[0]]

expected: 2

case 3

in: [[0.5,0.5],[0.25,0.75],[]]

expected: 1

case 4

in: [[0.5,0.5],[0.25,0.75],[1,1]]

expected: 0.4444444444444444

Starter codePython

def importance_ratio(target_probs, behavior_probs, actions):
    # Your code here
    pass

4 test cases · run your code against them in the editor