Chain Rule Entropy Check

Problem

Verify the entropy chain rule numerically. From a 2D joint distribution joint[x][y], return the absolute residual

|H(X, Y) - H(Y) - H(X|Y)|

where H(X|Y) is built from the row conditionals. The result should be within floating-point error of 0.

Hint

Compute H(X|Y) independently from the conditionals rather than reusing H(X, Y) - H(Y).

case 1

in: [[[0.25,0.25],[0.25,0.25]]]

expected: 0

case 2

in: [[[0.5,0],[0,0.5]]]

expected: 0

case 3

in: [[[0.1,0.2],[0.3,0.4]]]

expected: 0

case 4

in: [[[0.5,0.25],[0,0.25]]]

expected: 0

Starter codePython

import math
def chain_rule_gap(joint):
    # Your code here
    pass

4 test cases · run your code against them in the editor