MediumCalculusca-214

Bregman Divergence

Problem

For the negative entropy generator phi(x) = sum(x_i * log(x_i)), the Bregman divergence is the KL divergence:

D(p, q) = sum(p_i * log(p_i / q_i))

Terms with p_i = 0 contribute zero. Given probability vectors p and q, return the divergence.

Hint

KL divergence is nonnegative and equals zero only when the distributions match.

case 1

in: [[0.5,0.5],[0.5,0.5]]

expected: 0

case 2

in: [[0.5,0.5],[0.25,0.75]]

expected: 0.14384103622589042

case 3

in: [[1,0],[0.5,0.5]]

expected: 0.6931471805599453

case 4

in: [[0.25,0.75],[0.5,0.5]]

expected: 0.13081203594113697

Starter codePython

def bregman_kl(p, q):
    # Your code here
    pass

4 test cases · run your code against them in the editor