Label Smoothing Cross-Entropy

Problem

Cross-entropy with label smoothing for a single example: (1 - smoothing) * (-log p_target) + (smoothing / (num_classes - 1)) * sum over k != target of (-log p_k), where p is the softmax of the logits. Use a stable log-softmax.

The signature is label_smoothing_ce(logits, target, smoothing). Assume at least two classes.

Hint

Smoothing adds a small uniform target mass to every incorrect class, all sharing one coefficient.

Test cases

case 1

in: [[1,2,3],2,0]

expected: 0.4076059644

case 2

in: [[1,2,3],2,0.1]

expected: 0.5576059644

case 3

in: [[0,0],0,0.2]

expected: 0.6931471806

case 4

in: [[2,1,0],0,0.3]

expected: 0.8576059644

Starter codePython

import math
def label_smoothing_ce(logits, target, smoothing):
    # Your code here
    pass

4 test cases · run your code against them in the editor