HardNLPnlp-293

Temperature-Adjusted Entropy

Problem

Return the entropy in nats of softmax(logits / temperature). Higher temperature flattens the distribution and raises entropy, while lower temperature sharpens it.

Compute the softmax with the max-subtraction trick for stability, then entropy = -sum of p * log(p). Return 0.0 for empty logits. temperature is assumed positive.

Hint

Entropy is maximized by the uniform distribution and approaches 0 as one logit dominates.

Test cases

case 1

in: [[0,0],1]

expected: 0.6931471805599453

case 2

in: [[0,0],0.5]

expected: 0.6931471805599453

case 3

in: [[10,0],1]

expected: 0.0004993775862411646

case 4

in: [[10,0],10]

expected: 0.5822031088882179

case 5

in: [[],1]

expected: 0

Starter codePython

import math
def temperature_adjusted_entropy(logits, temperature):
    # Your code here
    pass

5 test cases · run your code against them in the editor