LR Finder Best Rate

Problem

Choose the best learning rate from an exponential LR-finder probe.

The probed rates are start_lr * (end_lr / start_lr)^(i / (num_steps - 1)) for i = 0 .. num_steps - 1. Return the probed rate at the index of the smallest loss, with the first minimum winning ties. If num_steps <= 1, return start_lr.

Hint

The probe grid is geometric, so index i maps to start_lr * ratio^i.

Test cases

case 1

in: [0.001,1,5,[2,1.5,0.8,1.2,3]]

expected: 0.0316227766016838

case 2

in: [0.01,0.1,3,[1,0.5,0.6]]

expected: 0.0316227766016838

case 3

in: [0.1,0.1,4,[1,1,1,1]]

expected: 0.1

case 4

in: [0.001,1,1,[5]]

expected: 0.001

Starter codePython

def lr_finder_best(start_lr, end_lr, num_steps, losses):
    # Your code here
    pass

4 test cases · run your code against them in the editor