SimPO Length-Normalized Reward

Problem

Compute the SimPO length-normalized reward margin for one pair.

Average the token log-probabilities inside each sequence, then return beta * (mean_chosen - mean_rejected) - gamma. If either sequence is empty return 0.0.

Hint

Dividing by sequence length removes the model's incentive to pad the preferred answer.

Test cases

case 1

in: [[-1,-1],[-2,-2],2,0.5]

expected: 1.5

case 2

in: [[-1],[-1],1,0.5]

expected: -0.5

case 3

in: [[-0.5,-1.5],[-2,-2],1,0]

expected: 1

case 4

in: [[],[],1,0]

expected: 0

Starter codePython

def simpo_reward(chosen_logps, rejected_logps, beta, gamma):
    # Your code here
    pass

4 test cases · run your code against them in the editor