Data Processing Inequality Gap

Problem

Check the data processing inequality for the Markov chain X -> Y -> Z. Given the joint pxy[x][y] and the channel channel[y][z] = P(z | y), compute I(X; Y) - I(X; Z) in bits, where Z is obtained by passing Y through the channel. The gap is always >= 0.

Hint

Post-processing Y into Z can never create information about X.

Test cases

case 1

in: [[[0.5,0],[0,0.5]],[[1,0],[0,1]]]

expected: 0

case 2

in: [[[0.5,0],[0,0.5]],[[0.5,0.5],[0.5,0.5]]]

expected: 1

case 3

in: [[[0.25,0.25],[0.25,0.25]],[[1,0],[0,1]]]

expected: 0

case 4

in: [[[0.4,0.1],[0.1,0.4]],[[0.9,0.1],[0.1,0.9]]]

expected: 0.10481827760525544

Starter codePython

import math
def data_processing_gap(pxy, channel):
    # Your code here
    pass

4 test cases · run your code against them in the editor