KL Divergence Asymmetry Check

Problem

Quantify how asymmetric KL divergence is for a pair of distributions by returning |D(p || q) - D(q || p)| in bits. Compute both divergences with 0 * log(0) treated as 0 and take the absolute difference.

Hint

KL is not symmetric, so the gap is generally positive.

Test cases

case 1

in: [[0.5,0.5],[0.25,0.75]]

expected: 0.018796874098554683

case 2

in: [[0.9,0.1],[0.5,0.5]]

expected: 0.20596118775548722

case 3

in: [[0.5,0.25,0.25],[0.25,0.5,0.25]]

expected: 0

case 4

in: [[0.25,0.25,0.25,0.25],[0.25,0.25,0.25,0.25]]

expected: 0

Starter codePython

import math
def kl_asymmetry(p, q):
    # Your code here
    pass

4 test cases · run your code against them in the editor