Group Norm Forward

Problem

Group normalization over channel groups. Split the channels into num_groups contiguous groups, compute the mean and biased variance over all features in each group, normalize, then scale by per-channel gamma and shift by per-channel beta.

The signature is group_norm_forward(x, gamma, beta, num_groups, eps=1e-5). x has shape (channels, features) and gamma/beta have length channels.

Hint

All channels in a group share one mean and variance; gamma and beta are still per channel.

Test cases

case 1

in: [[[1,2],[3,4]],[1,1],[0,0],2]

expected: [[-0.9999800006,0.9999800006],[-0.9999800006,0.9999800006]]

case 2

in: [[[1,2,3,4]],[1],[0],1]

expected: [[-1.34163542,-0.4472118067,0.4472118067,1.34163542]]

case 3

in: [[[1,1],[1,1]],[2,3],[0,0],1]

expected: [[0,0],[0,0]]

case 4

in: [[[1,2],[3,4]],[1,1],[1,0],2]

expected: [[0.0000199994,1.9999800006],[-0.9999800006,0.9999800006]]

Starter codePython

import math
def group_norm_forward(x, gamma, beta, num_groups, eps=1e-5):
    # Your code here
    pass

4 test cases · run your code against them in the editor