Layer Scale Apply

Problem

Apply LayerScale to a matrix: multiply each column j by the learned per-channel scale gamma[j], which is typically initialized near zero so residual branches start as identity.

The signature is layer_scale_apply(x, gamma). x has shape (rows, features) and gamma has length features.

Hint

This is a column-wise multiply by the gamma vector, applied to every row.

Test cases

case 1

in: [[[1,2],[3,4]],[0.1,0.2]]

expected: [[0.1,0.4],[0.3,0.8]]

case 2

in: [[[1,1],[1,1]],[1,1]]

expected: [[1,1],[1,1]]

case 3

in: [[[-1,2]],[0.5,-2]]

expected: [[-0.5,-4]]

case 4

in: [[[0,0],[5,6]],[2,3]]

expected: [[0,0],[10,18]]

Starter codePython

def layer_scale_apply(x, gamma):
    # Your code here
    pass

4 test cases · run your code against them in the editor