Entropy of Model Parameters

Problem

Model each parameter's posterior as an independent Gaussian with the given variance and return the total differential entropy in bits:

H = sum_i 0.5 * log2(2 * pi * e * var_i)

Hint

Uncertainty in the parameters is uncertainty about the model itself.

case 1

in: [[1]]

expected: 2.047095585180641

case 2

in: [[1,1]]

expected: 4.094191170361282

case 3

in: [[4,0.25]]

expected: 4.094191170361282

case 4

in: [[0.05854983152431917]]

expected: 0

Starter codePython

import math
def entropy_of_model_parameters(variances):
    # Your code here
    pass

4 test cases · run your code against them in the editor