Add Positional Encoding

Problem

Add positional encodings to a sequence of token embeddings.

The two matrices have the same shape. Return the element-wise sum as nested lists.

Hint

Position information is injected by simple addition before attention.

case 1

in: [[[1,2],[3,4]],[[0.5,0.5],[1,1]]]

expected: [[1.5,2.5],[4,5]]

case 2

in: [[[1,2,3]],[[-1,-2,-3]]]

expected: [[0,0,0]]

case 3

in: [[],[]]

expected: []

Starter codePython

def add_positional(token_embeddings, pe):
    # Your code here
    pass

3 test cases · run your code against them in the editor