Entropy of a Distribution

Problem

Compute the Shannon entropy in bits of a discrete probability distribution given as a list of probabilities:

H(p) = -sum(p_i * log2(p_i))

Skip zero-probability outcomes, treating 0 * log(0) as 0.

Hint

Entropy is maximized when all outcomes are equally likely.

case 1

in: [[0.5,0.5]]

expected: 1

case 2

in: [[1]]

expected: 0

case 3

in: [[0.25,0.25,0.25,0.25]]

expected: 2

case 4

in: [[0.5,0.25,0.25]]

expected: 1.5

case 5

in: [[0.7,0.3]]

expected: 0.8812908992306927

Starter codePython

import math
def entropy_bits(probs):
    # Your code here
    pass

5 test cases · run your code against them in the editor