Bahdanau Additive Attention

Problem

Compute Bahdanau additive attention scores: score_j = v dot tanh(W q + U k_j), where q is the query, k_j is the j-th key, W and U are projection matrices and v is the output vector. No softmax is applied.

The signature is bahdanau_scores(query, keys, W, U, v). Return the list of scores.

Hint

Additive attention projects query and key into a shared space, applies tanh, then scores with v.

Test cases

case 1

in: [[1,0],[[1,0],[0,1]],[[1,0],[0,1]],[[1,0],[0,1]],[1,0]]

expected: [0.9640275801,0.761594156]

case 2

in: [[1,1],[[1,0]],[[1,0],[0,1]],[[1,0],[0,1]],[1,1]]

expected: [1.725621736]

case 3

in: [[0,0],[[1,0],[0,1]],[[1,0],[0,1]],[[1,0],[0,1]],[1,0]]

expected: [0.761594156,0]

case 4

in: [[1,2],[[1,2]],[[1,0],[0,1]],[[1,0],[0,1]],[0,1]]

expected: [0.9993292997]

Starter codePython

import math
def bahdanau_scores(query, keys, W, U, v):
    # Your code here
    pass

4 test cases · run your code against them in the editor