Cosine vs Linear Schedule Compare

Problem

Compare a cosine schedule with a linear schedule at the same step.

Clamp progress = step / total to [0, 1], evaluate both schedules from base_lr to min_lr, and return cosine - linear. Return 0.0 when total is not positive.

Hint

The cosine curve is above the linear one early and below it later.

Test cases

case 1

in: [0.1,0,100,0]

expected: 0

case 2

in: [0.1,0,100,25]

expected: 0.010355339059327368

case 3

in: [0.1,0,100,50]

expected: 0

case 4

in: [0.1,0,100,100]

expected: 0

case 5

in: [0.1,0,0,5]

expected: 0

Starter codePython

def cosine_vs_linear_diff(base_lr, min_lr, total, step):
    # Your code here
    pass

5 test cases · run your code against them in the editor