Linear Warmup Learning Rate

Problem

Compute the learning rate during linear warmup:

lr = base_lr * min(1, max(0, step / warmup_steps))

If warmup_steps is not positive, return base_lr immediately.

Hint

The factor saturates at 1 after warmup_steps.

case 1

in: [0.1,10,0]

expected: 0

case 2

in: [0.1,10,5]

expected: 0.05

case 3

in: [0.1,10,20]

expected: 0.1

case 4

in: [0.1,0,7]

expected: 0.1

case 5

in: [0.2,4,-2]

expected: 0

Starter codePython

def linear_warmup_lr(base_lr, warmup_steps, step):
    # Your code here
    pass

5 test cases · run your code against them in the editor