Layer Normalization

Problem

Apply layer normalization to a single feature vector.

Compute the mean and population variance of x, then return (x_i - mean) / sqrt(var + eps) for each element. Use eps=1e-5 by default; a single element normalizes to 0.0 and an empty vector returns [].

Hint

Population variance divides by n, and eps keeps the denominator away from zero.

Test cases

case 1

in: [[1,2,3]]

expected: [-1.2247356859083902,0,1.2247356859083902]

case 2

in: [[0,0,0]]

expected: [0,0,0]

case 3

in: [[2,4]]

expected: [-0.9999950000374997,0.9999950000374997]

case 4

in: [[5]]

expected: [0]

Starter codePython

import math
def layer_norm(x, eps=1e-5):
    # Your code here
    pass

4 test cases · run your code against them in the editor