Mutual Information from Joint

Problem

Compute the mutual information I(X; Y) in bits from a 2D joint distribution joint[x][y]:

I = sum(p(x, y) * log2(p(x, y) / (p(x) * p(y))))

Skip terms with p(x, y) = 0.

Hint

Mutual information is the KL divergence between the joint and the product of marginals.

case 1

in: [[[0.25,0.25],[0.25,0.25]]]

expected: 0

case 2

in: [[[0.5,0],[0,0.5]]]

expected: 1

case 3

in: [[[0.1,0.2],[0.3,0.4]]]

expected: 0.005802149014345649

case 4

in: [[[0.5,0.25],[0.25,0]]]

expected: 0.12255624891826565

Starter codePython

import math
def mutual_information(joint):
    # Your code here
    pass

4 test cases · run your code against them in the editor