MediumNLPnlp-049

Softmax over Attention Scores

Problem

Return the softmax of a list of scores, computed with the max-subtraction trick: subtract the maximum score, exponentiate, and divide each value by the sum.

Return an empty list for empty input.

Hint

Subtracting the max keeps exp from overflowing without changing the result.

Test cases

case 1

in: [[1,2,3]]

expected: [0.09003057317038046,0.24472847105479764,0.6652409557748218]

case 2

in: [[0,0]]

expected: [0.5,0.5]

case 3

in: [[]]

expected: []

case 4

in: [[1000,1000]]

expected: [0.5,0.5]

case 5

in: [[-1,0,1]]

expected: [0.09003057317038046,0.24472847105479764,0.6652409557748218]

Starter codePython

import math
def softmax_scores(scores):
    # Your code here
    pass

5 test cases · run your code against them in the editor