MediumNLPnlp-376

Temperature Scaled Attention Weights

Problem

Dividing attention logits by a temperature T before softmax sharpens the distribution as T shrinks: softmax(scores / T).

Given the scores and temperature, return the weights computed stably.

Hint

All three softmax stages use the scaled scores.

case 1

in: [[1,2],1]

expected: [0.2689414213699951,0.7310585786300049]

case 2

in: [[1,2],0.5]

expected: [0.11920292202211755,0.8807970779778823]

case 3

in: [[0,0,0],2]

expected: [0.3333333333333333,0.3333333333333333,0.3333333333333333]

Starter codePython

def temperature_scaled_attention_weights(scores, temperature):
    # Your code here
    pass

3 test cases · run your code against them in the editor