GRPO Advantage

Problem

Compute group-relative policy optimization advantages: normalize a group of rewards to (r - mean) / std with the population standard deviation. If std is 0, return zeros.

The signature is grpo_advantage(rewards).

Hint

Using a group baseline instead of a learned value network is what makes GRPO lightweight.

Test cases

case 1

in: [[1,2,3,4]]

expected: [-1.3416407865,-0.4472135955,0.4472135955,1.3416407865]

case 2

in: [[5,5,5]]

expected: [0,0,0]

case 3

in: [[0,1]]

expected: [-1,1]

case 4

in: [[2,4,4,6]]

expected: [-1.4142135624,0,0,1.4142135624]

Starter codePython

import math
def grpo_advantage(rewards):
    # Your code here
    pass

4 test cases · run your code against them in the editor