HardNLPnlp-076

Cross-Attention Scores

Problem

Return the score matrix Q K^T / sqrt(d) for a list of query vectors and a list of key vectors, where d is the query dimension.

Element (i, j) is the dot product of queries[i] and keys[j] times the scaling factor. Return an empty list when either input is empty.

Hint

Compute every query-key dot product, then multiply the whole matrix by the scaling factor.

case 1

in: [[[1,0]],[[1,0],[0,1]]]

expected: [[0.7071067811865475,0]]

case 2

in: [[[1,1],[0,1]],[[1,0]]]

expected: [[0.7071067811865475],[0]]

case 3

in: [[[1]],[[2]]]

expected: [[2]]

case 4

in: [[],[[1]]]

expected: []

case 5

in: [[[1,0]],[]]

expected: []

Starter codePython

import math
def cross_attention_scores(queries, keys):
    # Your code here
    pass

5 test cases · run your code against them in the editor