Mutual Information (Discrete)

Problem

Compute the mutual information in bits between two discrete label lists: sum p(x, y) * log2(p(x, y) / (p(x) * p(y))) over all observed pairs with p(x, y) > 0.

Empty input returns 0.0.

Hint

MI is zero when the joint distribution factorizes into the product of the marginals.

Test cases

case 1

in: [[0,0,1,1],[0,0,1,1]]

expected: 1

case 2

in: [[0,1,0,1],[0,0,1,1]]

expected: 0

case 3

in: [[0,0,1,1],[0,1,1,1]]

expected: 0.31127812445913283

case 4

in: [[],[]]

expected: 0

Starter codePython

import math
def mutual_information(a, b):
    # Your code here
    pass

4 test cases · run your code against them in the editor