MediumNLPnlp-362

Scaled Dot Product Attention Weights

Problem

Scaled dot-product attention computes softmax(query dot key / sqrt(d_k)) over the keys.

Given the query and the list of keys, return the attention weights.

Hint

Scale by the square root of the key dimension before softmax.

case 1

in: [[1,0],[[1,0],[0,1]]]

expected: [0.6697615493266569,0.3302384506733431]

case 2

in: [[1,1],[[1,0],[0,1]]]

expected: [0.5,0.5]

case 3

in: [[1,0],[[2,0]]]

expected: [1]

Starter codePython

def scaled_dot_product_attention_weights(query, keys):
    # Your code here
    pass

3 test cases · run your code against them in the editor