Cross-Entropy with Smoothing

Problem

Return the cross-entropy in bits of a smoothed model. With n outcomes and smoothing eps, the model probability becomes

q'_i = (1 - eps) * q_i + eps / n

and the cross-entropy is -sum(p_i * log2(q'_i)) over outcomes with p_i > 0.

Hint

Smoothing lifts zero probabilities, so the cross-entropy stays finite.

case 1

in: [[0.5,0.5],[0.5,0.5],0.1]

expected: 1

case 2

in: [[1,0],[1,0],0.5]

expected: 0.4150374992788438

case 3

in: [[0.5,0.5],[0.25,0.75],0.2]

expected: 1.125769383497982

case 4

in: [[0.25,0.25,0.25,0.25],[1,0,0,0],0.4]

expected: 2.6200893643729612

Starter codePython

import math
def smoothed_cross_entropy(p, q, eps):
    # Your code here
    pass

4 test cases · run your code against them in the editor