Cross-Entropy of Gaussians

Problem

Return the differential cross-entropy in nats between two univariate Gaussians p = N(mu1, var1) and q = N(mu2, var2):

H(p, q) = 0.5 * ln(2 * pi * var2) + (var1 + (mu1 - mu2)^2) / (2 * var2)

Assume all variances are positive.

Hint

Cross-entropy equals the entropy of q plus KL(p || q).

case 1

in: [0,1,0,1]

expected: 1.4189385332046727

case 2

in: [1,1,0,1]

expected: 1.9189385332046727

case 3

in: [0,2,0,1]

expected: 1.9189385332046727

case 4

in: [0,1,0,4]

expected: 1.737085713764618

Starter codePython

import math
def cross_entropy_gaussians(mu1, var1, mu2, var2):
    # Your code here
    pass

4 test cases · run your code against them in the editor