Cross-Entropy from Counts

Problem

Given raw counts and a model distribution q, return the cross-entropy in bits of the empirical distribution:

H = -sum((c_i / sum(c)) * log2(q_i))

Skip outcomes with c_i = 0.

Hint

Normalize the counts into frequencies before weighting the log probabilities.

case 1

in: [[1,1],[0.5,0.5]]

expected: 1

case 2

in: [[3,1],[0.5,0.5]]

expected: 1

case 3

in: [[1,0],[0.25,0.75]]

expected: 2

case 4

in: [[1,2,3,4],[0.25,0.25,0.25,0.25]]

expected: 2

Starter codePython

import math
def cross_entropy_from_counts(counts, q):
    # Your code here
    pass

4 test cases · run your code against them in the editor