DDPG Noise Decay Schedule

Problem

Linearly decay an exploration noise standard deviation and floor it.

Return max(sigma_min, sigma0 - (sigma0 - sigma_min) * step / total_steps) with step clamped to [0, total_steps]. A non-positive total_steps returns sigma_min.

Hint

A nonzero noise floor keeps a deterministic policy exploring late in training.

Test cases

case 1

in: [0,0.5,0.05,100]

expected: 0.5

case 2

in: [100,0.5,0.05,100]

expected: 0.05

case 3

in: [50,0.5,0.05,100]

expected: 0.275

case 4

in: [200,0.5,0.05,100]

expected: 0.05

case 5

in: [10,0.5,0.6,100]

expected: 0.6

Starter codePython

def ddpg_noise_decay(step, sigma0, sigma_min, total_steps):
    # Your code here
    pass

5 test cases · run your code against them in the editor