Batch Scaling Rule Lite

Problem

Apply the square-root batch scaling rule.

Return base_lr * sqrt(new_batch / base_batch), which keeps the gradient noise per step roughly constant as the batch grows.

Hint

Quadrupling the batch doubles the recommended learning rate.

case 1

in: [0.1,32,128]

expected: 0.2

case 2

in: [0.01,100,100]

expected: 0.01

case 3

in: [0.1,16,4]

expected: 0.05

case 4

in: [0.2,64,1024]

expected: 0.8

Starter codePython

def batch_scaling_lr(base_lr, base_batch, new_batch):
    # Your code here
    pass

4 test cases · run your code against them in the editor