RMSNorm Forward

Problem

Apply RMSNorm: r = sqrt(mean(x^2) + eps), then out[i] = x[i] / r * weight[i]. Unlike LayerNorm there is no mean subtraction and no bias.

The signature is rmsnorm_forward(x, weight, eps=1e-6). Return a list of the same length as x.

Hint

Only the root mean square of the row matters; the mean is never subtracted.

case 1

in: [[3,4],[1,1],0]

expected: [0.8485281374,1.1313708499]

case 2

in: [[1,1],[2,3],0]

expected: [2,3]

case 3

in: [[0,0],[5,5],0.000001]

expected: [0,0]

case 4

in: [[1,2,3],[1,1,1],0]

expected: [0.4629100499,0.9258200998,1.3887301497]

Starter codePython

import math
def rmsnorm_forward(x, weight, eps=1e-6):
    # Your code here
    pass

4 test cases · run your code against them in the editor