Exponential Family Entropy

Problem

Return the entropy in nats of the exponential family p(t) proportional to exp(eta * t):

H = ln(Z) - eta * E[T]

where Z is the partition function and E[T] the mean of the sufficient statistic under p.

Hint

This is the Legendre transform relationship H = A - eta * A'(eta).

case 1

in: [0,[1,2,3]]

expected: 1.0986122886681098

case 2

in: [1,[0,1]]

expected: 0.5822031088882179

case 3

in: [2,[-1,0,1]]

expected: 0.4410574440581634

case 4

in: [-0.5,[0,2]]

expected: 0.582203108888218

Starter codePython

import math
def exponential_family_entropy(eta, t_values):
    # Your code here
    pass

4 test cases · run your code against them in the editor