KL Smoothing Penalty

Problem

Return the KL divergence in bits from p to its own eps-smoothed version:

p'_i = (1 - eps) * p_i + eps / n

D(p || p') is the information cost of adding eps of uniform smoothing.

Hint

Smoothing an already uniform distribution changes nothing.

case 1

in: [[1,0],0.5]

expected: 0.41503749927884376

case 2

in: [[0.5,0.5],0.25]

expected: 0

case 3

in: [[0.9,0.1],0.2]

expected: 0.03607129188493674

case 4

in: [[0.5,0.5],0]

expected: 0

Starter codePython

import math
def kl_smoothing_penalty(p, eps):
    # Your code here
    pass

4 test cases · run your code against them in the editor