KL Divergence of Gaussians

Problem

Return the KL divergence in nats between two univariate Gaussians:

D = 0.5 * (ln(var2 / var1) + (var1 + (mu1 - mu2)^2) / var2 - 1)

Use the natural logarithm. Assume all variances are positive.

Hint

Match means or variances and the formula reduces to familiar special cases.

case 1

in: [0,1,0,1]

expected: 0

case 2

in: [1,1,0,1]

expected: 0.5

case 3

in: [0,1,0,4]

expected: 0.3181471805599453

case 4

in: [0,2,0,1]

expected: 0.1534264097200273

case 5

in: [3,2,1,4]

expected: 0.5965735902799727

Starter codePython

import math
def kl_gaussian(mu1, var1, mu2, var2):
    # Your code here
    pass

5 test cases · run your code against them in the editor