Entropy Calibration Loss

Problem

Compute the cross-entropy calibration loss: normalize both histograms, floor the quantized probabilities at eps, and return -sum p_i * ln(q_i). This is the entropy of the reference measured under the quantized distribution.

The signature is entropy_calibration_loss(reference, quantized, eps=1e-9).

Hint

Cross-entropy is never below the reference entropy; matching histograms hit that floor.

Test cases

case 1

in: [[0.5,0.5],[0.5,0.5]]

expected: 0.6931471805599453

case 2

in: [[1,0],[1,0]]

expected: 0

case 3

in: [[1,0],[0.5,0.5]]

expected: 0.6931471805599453

case 4

in: [[0.25,0.75],[0.5,0.5]]

expected: 0.6931471805599453

Starter codePython

import math
def entropy_calibration_loss(reference, quantized, eps=1e-9):
    # Your code here
    pass

4 test cases · run your code against them in the editor