KL Term in ELBO

Problem

The KL divergence from a discrete variational distribution q to a prior p is the sum over states of q[i] * log(q[i] / p[i]). Return it in nats, skipping states with q[i] equal to zero. Return 0.0 for mismatched or empty inputs, negative entries, or a state with q[i] positive and p[i] equal to zero.

Hint

The KL is the sum of q log(q / p), with 0 log 0 treated as zero.

Test cases

case 1

in: [[0.5,0.5],[0.5,0.5]]

expected: 0

case 2

in: [[0.75,0.25],[0.5,0.5]]

expected: 0.13081203594113697

case 3

in: [[1,0],[0.5,0.5]]

expected: 0.6931471805599453

case 4

in: [[0,1],[0.5,0.5]]

expected: 0.6931471805599453

case 5

in: [[0.5,0.5],[0,1]]

expected: 0

Starter codePython

import math


def kl_term_in_elbo(q, p):
    # Your code here
    pass

5 test cases · run your code against them in the editor