PPO Minibatch Advantage Normalization

Problem

Standardize PPO advantages inside a minibatch with (a - mean) / (population_std + epsilon). If every advantage is equal, or the batch has one element, return zeros; an empty batch returns [].

Hint

Normalizing per minibatch keeps the effective learning rate steady as advantage scales drift.

Test cases

case 1

in: [[1,2,3],1e-8]

expected: [-1.2247448563915893,0,1.2247448563915893]

case 2

in: [[5,5],1e-8]

expected: [0,0]

case 3

in: [[],1e-8]

expected: []

case 4

in: [[2],1e-8]

expected: [0]

Starter codePython

import math
def ppo_standardize_advantages(advantages, epsilon):
    # Your code here
    pass

4 test cases · run your code against them in the editor