Expected Calibration Error Bins

Problem

Expected calibration error is the weighted average over confidence bins of |accuracy_b - confidence_b|, weighted by the fraction of samples in each bin.

Given the bin confidences, bin accuracies, and bin fractions, return the ECE.

Hint

Weight each bin gap by its sample share.

Test cases

case 1

in: [[0.1,0.9],[0.2,0.8],[0.5,0.5]]

expected: 0.09999999999999999

case 2

in: [[0.5,0.5],[0.5,0.5],[0.5,0.5]]

expected: 0

case 3

in: [[0.2,0.8],[0.1,0.9],[0.25,0.75]]

expected: 0.09999999999999998

Starter codePython

def expected_calibration_error(confidences, accuracies, fractions):
    # Your code here
    pass

3 test cases · run your code against them in the editor