GRPO Group Advantage Normalization

Problem

Normalize the rewards inside every group of a GRPO batch.

For each group compute (r - group_mean) / group_population_std. Groups with fewer than two samples or zero standard deviation become zeros, and an empty group stays empty.

Hint

GRPO drops the value network and uses the group itself as the baseline.

Test cases

case 1

in: [[[1,2,3]]]

expected: [[-1.224744871391589,0,1.224744871391589]]

case 2

in: [[[1,1,1],[0,2]]]

expected: [[0,0,0],[-1,1]]

case 3

in: [[[5],[]]]

expected: [[0],[]]

case 4

in: [[]]

expected: []

Starter codePython

import math
def grpo_group_advantages(group_rewards):
    # Your code here
    pass

4 test cases · run your code against them in the editor