KL Divergence Between Gaussians

Problem

Kullback-Leibler divergence from N(mu1, sigma1^2) to N(mu2, sigma2^2): ln(sigma2 / sigma1) + (sigma1^2 + (mu1 - mu2)^2) / (2 * sigma2^2) - 0.5. Return 0.0 when either standard deviation is nonpositive.

Hint

The formula combines the log variance ratio, the variance ratio and the squared mean shift.

Test cases

case 1

in: [0,1,0,1]

expected: 0

case 2

in: [0,1,1,1]

expected: 0.5

case 3

in: [0,2,0,1]

expected: 0.8068528194400546

case 4

in: [1,1,0,2]

expected: 0.4431471805599453

case 5

in: [0,1,0,-1]

expected: 0

Starter codePython

import math


def kl_gaussian(mu1, sigma1, mu2, sigma2):
    # Your code here
    pass

5 test cases · run your code against them in the editor