Maximum Entropy with Mean Constraint

Problem

Over the support {0, 1, 2}, find the maximum-entropy distribution whose mean equals mu. The solution is the exponential family p(x) proportional to exp(theta * x); find theta by bisection on [-60, 60] for 200 iterations so the mean matches mu, then return the entropy in bits. Assume 0 < mu < 2.

Hint

The mean is increasing in theta, so bisection converges; mu = 1 gives the uniform distribution.

Test cases

case 1

in: [1]

expected: 1.584962500721156

case 2

in: [0.5]

expected: 1.3002068332819543

case 3

in: [1.5]

expected: 1.300206833281954

case 4

in: [0.2]

expected: 0.7726396477406536

Starter codePython

import math
def max_entropy_mean(mu):
    # Your code here
    pass

4 test cases · run your code against them in the editor