Attention Decoder Context

Problem

Compute the context vector for an attention decoder: score each encoder state with a dot product against the decoder query, apply a stable softmax over the scores, then take the weighted sum of the encoder states.

The signature is attention_context(query, encoder_states). Return a vector with the same dimension as the query.

Hint

One encoder state means the softmax is 1.0 and the context is exactly that state.

Test cases

case 1

in: [[1,0],[[1,0],[0,1]]]

expected: [0.7310585786,0.2689414214]

case 2

in: [[0,1],[[1,0],[0,1]]]

expected: [0.2689414214,0.7310585786]

case 3

in: [[1,1],[[1,0],[0,1]]]

expected: [0.5,0.5]

case 4

in: [[2,0],[[3,4]]]

expected: [3,4]

Starter codePython

import math
def attention_context(query, encoder_states):
    # Your code here
    pass

4 test cases · run your code against them in the editor