Mutual Information from Counts

Problem

Compute the mutual information I(X; Y) in bits from a 2D table of counts, summing p(x, y) log2(p(x, y) / (p(x) p(y))). Zero rows, zero columns, and zero cells are skipped. Return 0.0 for an empty or all-zero table.

Hint

Divide each joint probability by the product of its marginals before taking the logarithm.

Test cases

case 1

in: [[[1,1],[1,1]]]

expected: 0

case 2

in: [[[2,0],[0,2]]]

expected: 1

case 3

in: [[[1,2],[3,4]]]

expected: 0.005802149014345795

case 4

in: [[[4,4],[4,4]]]

expected: 0

case 5

in: [[[0,0],[0,0]]]

expected: 0

Starter codePython

import math
def mutual_information_counts(counts):
    # Your code here
    pass

5 test cases · run your code against them in the editor