Expected Information from a Test

Problem

Compute the expected information in bits that an experiment provides about a hypothesis, which equals the mutual information between hypothesis and observation. For each observation, form the posterior from the prior and the likelihoods, then accumulate P(observation) times the KL divergence from prior to posterior.

Hint

Weight each posterior surprise by the probability of seeing that observation.

Test cases

case 1

in: [[0.5,0.5],[[0.9,0.1],[0.2,0.8]]]

expected: 0.3973126097494865

case 2

in: [[0.5,0.5],[[0.5,0.5],[0.5,0.5]]]

expected: 0

case 3

in: [[0.8,0.2],[[0.9,0.1],[0.1,0.9]]]

expected: 0.35775077890333656

case 4

in: [[0.25,0.75],[[0.8,0.2],[0.4,0.6]]]

expected: 0.09130503043715799

Starter codePython

import math
def expected_information(prior, likelihoods):
    # Your code here
    pass

4 test cases · run your code against them in the editor