Bayesian Surprise of an Observation

Problem

Compute the Bayesian surprise, the KL divergence in bits from the prior to the posterior after observing one data point. likelihoods[i][o] is the probability of observation o under hypothesis i. Return 0.0 when the posterior cannot be normalized or the prior has zero mass.

Hint

Multiply the prior by the likelihoods, normalize, then measure the divergence from the prior.

Test cases

case 1

in: [[0.5,0.5],[[0.9,0.1],[0.2,0.8]],0]

expected: 0.3159615643609581

case 2

in: [[0.5,0.5],[[0.9,0.1],[0.2,0.8]],1]

expected: 0.4967416652243545

case 3

in: [[0.3,0.7],[[0.5,0.5],[0.5,0.5]],0]

expected: 0

case 4

in: [[0.8,0.2],[[0.9,0.1],[0.1,0.9]],1]

expected: 0.8160518392832559

Starter codePython

import math
def bayesian_surprise(prior, likelihoods, observation):
    # Your code here
    pass

4 test cases · run your code against them in the editor