Jensen Gap of Entropy

Problem

Return the Jensen gap of the entropy function for the mixture w*p + (1-w)*q, that is the entropy of the mixture minus the same mixture of the individual entropies, measured in bits. Entropy is concave, so the gap is nonnegative and tiny negative rounding noise is clamped to 0.0.

Hint

Mix the distributions first, then compare H(mixture) with the weighted average of H(p) and H(q).

Test cases

case 1

in: [[1,0],[0,1],0.5]

expected: 1

case 2

in: [[1,0],[1,0],0.3]

expected: 0

case 3

in: [[1,0],[0,1],0.25]

expected: 0.8112781244591328

case 4

in: [[0.5,0.5],[0.25,0.75],0.5]

expected: 0.04879494069539858

case 5

in: [[0.5,0.5],[0.5,0.5],0.4]

expected: 0

Starter codePython

import math
def jensen_gap(p, q, w):
    # Your code here
    pass

5 test cases · run your code against them in the editor