KL Divergence from Counts

Problem

Normalize two count lists into distributions and return the KL divergence D(p||q) in bits. Zero entries of p are skipped and mismatched lengths are truncated to the shorter list. Return 0.0 if either total is zero.

Hint

Normalize both count lists first, then sum p*log2(p/q) over the entries of p.

Test cases

case 1

in: [[1,1],[1,1]]

expected: 0

case 2

in: [[1,1],[1,3]]

expected: 0.20751874963942185

case 3

in: [[2,1,1],[1,1,2]]

expected: 0.25

case 4

in: [[3,0],[1,1]]

expected: 1

case 5

in: [[0,0],[1,1]]

expected: 0

Starter codePython

import math
def kl_from_counts(p_counts, q_counts):
    # Your code here
    pass

5 test cases · run your code against them in the editor