Mutual Information of Independent Variables

Problem

Build the joint distribution p(x, y) = px[x] * py[y] of two independent variables and return their mutual information in bits. The answer should be 0; the point is to confirm it numerically for the product distribution.

Hint

Independence means p(x, y) = p(x) * p(y), so every log-ratio is exactly zero.

Test cases

case 1

in: [[0.5,0.5],[0.5,0.5]]

expected: 0

case 2

in: [[0.25,0.75],[0.1,0.9]]

expected: 0

case 3

in: [[0.2,0.3,0.5],[0.5,0.5]]

expected: 0

case 4

in: [[1],[0.4,0.6]]

expected: 0

Starter codePython

import math
def mutual_information_independent(px, py):
    # Your code here
    pass

4 test cases · run your code against them in the editor