A2C Advantage Normalization

Problem

Normalize a batch of advantages to zero mean and unit variance.

Use the population standard deviation (divide by n) and add epsilon to the denominator for stability. Return [] for an empty batch, and return zeros when the standard deviation is below epsilon.

Hint

A2C standardizes the advantage batch so the policy gradient has a stable scale.

Test cases

case 1

in: [[1,2,3]]

expected: [-1.2247448563915893,0,1.2247448563915893]

case 2

in: [[5,5,5]]

expected: [0,0,0]

case 3

in: [[]]

expected: []

case 4

in: [[-1,0,1,2]]

expected: [-1.341640774499874,-0.447213591499958,0.447213591499958,1.341640774499874]

Starter codePython

def normalize_advantages(advantages, epsilon=1e-8):
    # Your code here
    pass

4 test cases · run your code against them in the editor