Directed Information Lite

Problem

Compute the one-step directed information in bits over the joint joint[x0][y0][x1][y1]:

DI = I(X0, X1; Y1 | Y0) = H(X0, X1 | Y0) - H(X0, X1 | Y0, Y1)

Marginalize the 4D table to build both conditional entropies.

Hint

Condition on Y0 by subtracting H(Y0) from the joint entropies of the remaining variables.

case 1

in: [[[[[0.0625,0.0625],[0.0625,0.0625]],[[0.0625,0.0625],[0.0625,0.0625]]],[[[0.0625,0.0625],[0.0625,0.0625]],[[0.0625,0.0625],[0.0625,0.0625]]]]]

expected: 0

case 2

in: [[[[[0.125,0],[0.125,0]],[[0.125,0],[0.125,0]]],[[[0,0.125],[0,0.125]],[[0,0.125],[0,0.125]]]]]

expected: 1

case 3

in: [[[[[0.125,0],[0,0.125]],[[0.125,0],[0,0.125]]],[[[0.125,0],[0,0.125]],[[0.125,0],[0,0.125]]]]]

expected: 1

case 4

in: [[[[[0.1,0.025],[0.1,0.025]],[[0.1,0.025],[0.1,0.025]]],[[[0.025,0.1],[0.025,0.1]],[[0.025,0.1],[0.025,0.1]]]]]

expected: 0.27807190511263835

Starter codePython

import math
def directed_information_lite(joint):
    # Your code here
    pass

4 test cases · run your code against them in the editor