Reward Function From Preferences

Problem

Estimate linear reward weights from pairwise trajectory preferences.

Each pair is [phi1, phi2, pref], where pref >= 0.5 means phi1 is preferred. Add sign * (phi1 - phi2) for every pair and divide by the number of pairs. Return the weight vector, or [] when there are no pairs.

Hint

Preferences identify reward weights only up to scale, so the sign convention matters.

Test cases

case 1

in: [[[[1,2],[0,1],1],[[2,0],[1,1],0]]]

expected: [0,1]

case 2

in: [[[[1],[0],1],[[1],[2],1]]]

expected: [0]

case 3

in: [[[[3,1],[0,2],0.5]]]

expected: [3,-1]

Starter codePython

def reward_from_preferences(pairs):
    # Your code here
    pass

3 test cases · run your code against them in the editor