Total Correlation

Problem

Compute the total correlation (multi-information) in bits of a 3D joint distribution joint[x][y][z]:

TC = H(X) + H(Y) + H(Z) - H(X, Y, Z)

Hint

Total correlation is zero exactly when the three variables are independent.

case 1

in: [[[[0.125,0.125],[0.125,0.125]],[[0.125,0.125],[0.125,0.125]]]]

expected: 0

case 2

in: [[[[0.5,0],[0,0]],[[0,0],[0,0.5]]]]

expected: 2

case 3

in: [[[[0.25,0.25],[0,0]],[[0,0],[0.25,0.25]]]]

expected: 1

case 4

in: [[[[0.2,0.1],[0.05,0.05]],[[0.05,0.05],[0.1,0.2]]]]

expected: 0.715084951819779

Starter codePython

import math
def total_correlation(joint):
    # Your code here
    pass

4 test cases · run your code against them in the editor