Bregman Divergence in Information Geometry

Problem

Return the Bregman divergence generated by f(x) = x ln(x) applied to two discrete distributions p and q:

D(p, q) = sum_i (p_i * ln(p_i / q_i) - p_i + q_i)

Skip the log term when p_i = 0. Assume q_i > 0 wherever p_i > 0.

Hint

The negentropy generator makes the Bregman divergence equal to KL plus a correction.

case 1

in: [[0.5,0.5],[0.5,0.5]]

expected: 0

case 2

in: [[1,0],[0.5,0.5]]

expected: 0.6931471805599453

case 3

in: [[0.25,0.75],[0.5,0.5]]

expected: 0.130812035941137

case 4

in: [[0.5,0.5],[0.25,0.75]]

expected: 0.14384103622589042

Starter codePython

import math
def bregman_divergence(p, q):
    # Your code here
    pass

4 test cases · run your code against them in the editor