Softmax + Cross-Entropy Backward

Problem

Gradient of the cross-entropy loss with a softmax output with respect to the logits: p - one_hot(label), where p = softmax(logits).

The signature is softmax_ce_backward(logits, label). Return the gradient vector.

Hint

The softmax and cross-entropy derivatives cancel beautifully: subtract 1 from the target class probability.

case 1

in: [[1,2,3],2]

expected: [0.0900305732,0.2447284711,-0.3347590442]

case 2

in: [[0,0],0]

expected: [-0.5,0.5]

case 3

in: [[5,5,5],1]

expected: [0.3333333333,-0.6666666667,0.3333333333]

case 4

in: [[1000,1000],1]

expected: [0.5,-0.5]

Starter codePython

import math
def softmax_ce_backward(logits, label):
    # Your code here
    pass

4 test cases · run your code against them in the editor