Reference-Free Preference Eval

Problem

Evaluate a policy without a reference model by averaging its chosen-minus-rejected log-probability margins: mean(chosen_logps[i] - rejected_logps[i]). An empty batch returns 0.0.

Hint

Reference-free metrics avoid running a second model at evaluation time.

Test cases

case 1

in: [[-1,-2],[-2,-1]]

expected: 0

case 2

in: [[-1],[-3]]

expected: 2

case 3

in: [[],[]]

expected: 0

Starter codePython

def reference_free_margin(chosen_logps, rejected_logps):
    # Your code here
    pass

3 test cases · run your code against them in the editor