Softmax Cross-Entropy Gradient

Problem

Return the gradient of the softmax cross-entropy loss with respect to a single sample's logits as a list. With a stable softmax p the gradient is p with 1 subtracted at the true class index. Use the max-subtraction trick for numerical stability.

Hint

Softmax plus cross-entropy differentiates to predicted minus one-hot.

Test cases

case 1

in: [[2,1,0],0]

expected: [-0.3347590442251782,0.24472847105479764,0.09003057317038046]

case 2

in: [[0,0],1]

expected: [0.5,-0.5]

case 3

in: [[1,2,3],2]

expected: [0.09003057317038046,0.24472847105479764,-0.3347590442251782]

case 4

in: [[5],0]

expected: [0]

case 5

in: [[0,0,0,0],3]

expected: [0.25,0.25,0.25,-0.75]

Starter codePython

import math
def softmax_ce_gradient(logits, label):
    # Your code here
    pass

5 test cases · run your code against them in the editor