Cyclical Learning Rate

Problem

Compute a triangular cyclical learning rate.

With cycle = 1 + step // (2 * step_size) and x = abs(step / step_size - 2 * cycle + 1), return base_lr + (max_lr - base_lr) * max(0, 1 - x). If step_size is not positive, return base_lr.

Hint

The cycle rises during the first half and falls during the second.

Test cases

case 1

in: [0.001,0.01,5,0]

expected: 0.001

case 2

in: [0.001,0.01,5,5]

expected: 0.010000000000000002

case 3

in: [0.001,0.01,5,2]

expected: 0.0046

case 4

in: [0.001,0.01,5,7]

expected: 0.006400000000000001

case 5

in: [0.001,0.01,5,10]

expected: 0.001

Starter codePython

def cyclical_lr(base_lr, max_lr, step_size, step):
    # Your code here
    pass

5 test cases · run your code against them in the editor