SAM Perturbation Norm

Problem

Compute the norm of a Sharpness-Aware Minimization perturbation.

Normalizing the gradient by its Euclidean norm gives a perturbation rho * g / (||g|| + eps); return its norm, which equals rho * ||g|| / (||g|| + eps). A zero gradient yields 0.0.

Hint

The perturbation has norm just under rho when the gradient is large.

Test cases

case 1

in: [[3,4],0.05,1e-8]

expected: 0.0499999999

case 2

in: [[0,0],0.1,1e-8]

expected: 0

case 3

in: [[1,1,1,1],0.2,1e-8]

expected: 0.199999999

case 4

in: [[1,0],0.1,1]

expected: 0.05

Starter codePython

def sam_perturbation_norm(grad, rho, eps):
    # Your code here
    pass

4 test cases · run your code against them in the editor