Entropy of a Distribution Mixture

Problem

Compute the Shannon entropy in bits of the mixture w*p + (1-w)*q of two probability distributions. Zero mixture entries are skipped. Assume p and q have equal length and each sums to one.

Hint

Average the two distributions entry by entry, then take the usual entropy.

Test cases

case 1

in: [[1,0],[0,1],0.5]

expected: 1

case 2

in: [[1,0],[1,0],0.3]

expected: 0

case 3

in: [[1,0],[0,1],0.9]

expected: 0.4689955935892811

case 4

in: [[0.5,0.5],[0.25,0.75],0.5]

expected: 0.954434002924965

case 5

in: [[0.5,0.5],[0.5,0.5],0]

expected: 1

Starter codePython

import math
def mixture_entropy(p, q, w):
    # Your code here
    pass

5 test cases · run your code against them in the editor