Interaction Information

Problem

Compute the interaction information in bits from a 3D joint distribution joint[x][y][z]:

II = I(X; Y) - I(X; Y | Z)

Use the marginal p(x, y) for the first term. The result can be negative.

Hint

Interaction information is symmetric in X, Y, and Z even though this formula does not look symmetric.

case 1

in: [[[[0.125,0.125],[0.125,0.125]],[[0.125,0.125],[0.125,0.125]]]]

expected: 0

case 2

in: [[[[0.5,0],[0,0]],[[0,0],[0,0.5]]]]

expected: 1

case 3

in: [[[[0.25,0.25],[0,0]],[[0,0],[0.25,0.25]]]]

expected: 0

case 4

in: [[[[0.2,0.1],[0.05,0.05]],[[0.05,0.05],[0.1,0.2]]]]

expected: 0.2814255716626396

Starter codePython

import math
def interaction_information(joint):
    # Your code here
    pass

4 test cases · run your code against them in the editor