Predictive Information

Problem

Compute the predictive information in bits from the joint joint[x_prev][x_cur][x_next]:

PI = I((X_prev, X_cur); X_next) = H(X_next) - H(X_next | X_prev, X_cur)

Hint

Predictive information measures how much the two-step history tells us about the future.

case 1

in: [[[[0.125,0.125],[0.125,0.125]],[[0.125,0.125],[0.125,0.125]]]]

expected: 0

case 2

in: [[[[0.25,0],[0,0.25]],[[0.25,0],[0,0.25]]]]

expected: 1

case 3

in: [[[[0.25,0],[0,0.25]],[[0,0.25],[0.25,0]]]]

expected: 1

case 4

in: [[[[0.2,0.05],[0.05,0.2]],[[0.2,0.05],[0.05,0.2]]]]

expected: 0.2780719051126379

Starter codePython

import math
def predictive_information(joint):
    # Your code here
    pass

4 test cases · run your code against them in the editor