Weighted Bradley-Terry Reward Loss

Problem

Compute a weighted Bradley-Terry reward-model loss.

Each pair contributes weight[i] * softplus(-(r_pref - r_rej)), using the stable form max(y, 0) + log1p(exp(-|y|)) with y = -(r_pref - r_rej), and the total is divided by the sum of the weights. Return 0.0 when the batch is empty or all weights are zero.

Hint

Sample weights let you downweight noisy or low-confidence preference pairs.

Test cases

case 1

in: [[0,0],[0,0],[1,1]]

expected: 0.6931471805599453

case 2

in: [[1,0],[0,0],[1,3]]

expected: 0.5981758072995147

case 3

in: [[1],[0],[0]]

expected: 0

case 4

in: [[],[],[]]

expected: 0

Starter codePython

import math
def weighted_bt_loss(r_pref, r_rej, weights):
    # Your code here
    pass

4 test cases · run your code against them in the editor