Posterior Entropy

Problem

Given unnormalized log weights for a discrete posterior, normalize them with the log-sum-exp trick and return the Shannon entropy -sum of p * log(p) in nats. Return 0.0 for an empty list.

Hint

Subtract the maximum log weight before exponentiating to keep the normalization stable.

Test cases

case 1

in: [[0,0]]

expected: 0.6931471805599453

case 2

in: [[0,1.0986122886681098]]

expected: 0.5623351446188083

case 3

in: [[0,0,0,0]]

expected: 1.3862943611198906

case 4

in: [[-0.10536051565782628,-2.302585092994046]]

expected: 0.32508297339144837

case 5

in: [[]]

expected: 0

Starter codePython

import math


def posterior_entropy(log_weights):
    # Your code here
    pass

5 test cases · run your code against them in the editor