Effective Information

Problem

Return the effective information in bits of a channel channel[x][y] under a maximum-entropy (uniform) input distribution. Compute the output marginal p(y) and average the KL divergence of each row from that marginal:

EI = (1 / n) * sum_x KL(P(Y | x) || p(Y))

Hint

Effective information is the mutual information the mechanism generates from noise.

Test cases

case 1

in: [[[1,0],[0,1]]]

expected: 1

case 2

in: [[[0.9,0.1],[0.1,0.9]]]

expected: 0.5310044064107189

case 3

in: [[[0.5,0.5],[0.5,0.5]]]

expected: 0

case 4

in: [[[0.75,0.25,0],[0,0.25,0.75]]]

expected: 0.75

Starter codePython

import math
def effective_information(channel):
    # Your code here
    pass

4 test cases · run your code against them in the editor