Warmup Heuristic Length

Problem

Compute the recommended warmup length for Adam from the second-moment decay.

Return ceil(2 / (1 - beta2)) steps, the heuristic that lets the second moment approach its stationary value. Return 0 when beta2 is outside [0, 1).

Hint

Larger beta2 means a longer memory and a longer recommended warmup.

case 1

in: [0.999]

expected: 2000

case 2

in: [0.9]

expected: 21

case 3

in: [0.99]

expected: 200

case 4

in: [1]

expected: 0

Starter codePython

def warmup_heuristic_length(beta2):
    # Your code here
    pass

4 test cases · run your code against them in the editor