Gradient Clipping By Norm

Problem

Clip a gradient vector by its global L2 norm.

Compute norm = sqrt(sum(g * g)). If norm > max_norm and norm > 0, rescale every component by max_norm / norm. Otherwise return the original values unchanged. This preserves the gradient direction.

Hint

If the norm is already within max_norm, return the vector untouched.

Test cases

case 1

in: [[3,4],2.5]

expected: [1.5,2]

case 2

in: [[0.1,0.2],5]

expected: [0.1,0.2]

case 3

in: [[0,0],1]

expected: [0,0]

case 4

in: [[-3,4],10]

expected: [-3,4]

case 5

in: [[1,0],1]

expected: [1,0]

Starter codePython

def clip_by_norm(grads, max_norm):
    # Your code here
    pass

5 test cases · run your code against them in the editor