MI Chain Rule Gap

Problem

Verify the mutual information chain rule numerically. From a 3D joint joint[x][y][z], return the absolute residual

|I(X; Y,Z) - I(X; Y) - I(X; Z | Y)|

All quantities are in bits and the residual should be within floating-point error of 0.

Hint

I(X; Z | Y) = sum p(x, y, z) log2(p(x, y, z) * p(y) / (p(x, y) * p(y, z))).

case 1

in: [[[[0.125,0.125],[0.125,0.125]],[[0.125,0.125],[0.125,0.125]]]]

expected: 0

case 2

in: [[[[0.5,0],[0,0]],[[0,0],[0,0.5]]]]

expected: 0

case 3

in: [[[[0.25,0.25],[0,0]],[[0,0],[0.25,0.25]]]]

expected: 0

case 4

in: [[[[0.2,0.1],[0.05,0.05]],[[0.05,0.05],[0.1,0.2]]]]

expected: 0

Starter codePython

import math
def mi_chain_rule_gap(joint):
    # Your code here
    pass

4 test cases · run your code against them in the editor