Cross-Entropy with Label Smoothing

Problem

Compute the mean cross-entropy loss in nats when one-hot targets are smoothed toward the uniform distribution. For K classes each target becomes 1-eps at the true class and eps/K elsewhere, and log probabilities are floored at 1e-12. Return 0.0 for empty input.

Hint

Every class contributes, so sum the smoothed target times the negative log probability over all classes.

Test cases

case 1

in: [[[0.8,0.2]],[0],0]

expected: 0.2231435513142097

case 2

in: [[[0.8,0.2]],[0],0.1]

expected: 0.29245826937020425

case 3

in: [[[0.9,0.1]],[1],0.2]

expected: 2.0828626352604234

case 4

in: [[[0.6,0.4],[0.5,0.5]],[0,1],0.1]

expected: 0.6121230298656721

case 5

in: [[],[],0]

expected: 0

Starter codePython

import math
def smoothed_cross_entropy(probs, labels, eps):
    # Your code here
    pass

5 test cases · run your code against them in the editor