Intrinsic Reward Running Average

Problem

Update running statistics of intrinsic rewards with Welford's method and return the normalized reward.

Given the mean, m2, and count before the reward, compute the new mean and m2, take std = sqrt(m2 / count), and normalize as reward / (1 + std). Return [normalized, new_mean, new_m2, count + 1].

Hint

Normalizing by the running standard deviation keeps intrinsic rewards at a stable scale.

Test cases

case 1

in: [0,0,0,5]

expected: [5,5,0,1]

case 2

in: [5,0,1,3]

expected: [1.5,4,2,2]

case 3

in: [4,2,2,7]

expected: [2.658571279792899,5,8,3]

case 4

in: [2,1,3,2]

expected: [1.3333333333333333,2,1,4]

Starter codePython

import math
def intrinsic_reward_update(mean, m2, count, reward):
    # Your code here
    pass

4 test cases · run your code against them in the editor