MediumCalculusca-192

KL Divergence Derivative

Problem

The KL divergence between univariate Gaussians q = N(mu1, sigma1^2) and p = N(mu2, sigma2^2) is

KL = log(sigma2 / sigma1) + (sigma1^2 + (mu1 - mu2)^2) / (2 * sigma2^2) - 0.5

Return its derivatives with respect to the parameters of q as [dKL/dmu1, dKL/dsigma1]. The term (mu1 - mu2)/sigma2^2 and the sigma1 term follow by differentiation.

Hint

Differentiate the closed form with respect to each parameter while holding the other fixed.

Test cases

case 1

in: [0,1,0,1]

expected: [0,0]

case 2

in: [1,1,0,1]

expected: [1,0]

case 3

in: [0,2,0,1]

expected: [0,1.5]

case 4

in: [3,0.5,1,2]

expected: [0.5,-1.875]

Starter codePython

def kl_gaussian_grad(mu1, sigma1, mu2, sigma2):
    # Return [dKL_dmu1, dKL_dsigma1]
    # Your code here
    pass

4 test cases · run your code against them in the editor