IPS Off-Policy Estimate

Problem

Compute the inverse propensity scoring off-policy estimate: mean(target_prob / behavior_prob * reward), clipping the behavior probability to at least 1e-6.

Empty input returns 0.0.

Hint

IPS reweights logged rewards by how much more likely the target policy was to act.

Test cases

case 1

in: [[1,0,2],[0.5,0.5,0.4],[0.7,0.3,0.5]]

expected: 1.3

case 2

in: [[1],[0.25],[1]]

expected: 4

case 3

in: [[],[],[]]

expected: 0

Starter codePython

def ips_estimate(rewards, behavior_probs, target_probs):
    # Your code here
    pass

3 test cases · run your code against them in the editor