MediumCalculusca-345

Softmax Cross-Entropy Logit Gradient

Problem

For the one-hot cross-entropy loss with softmax outputs, the gradient with respect to the logits is p - y, where p = softmax(logits).

Given the logits and the correct class index, return the gradient vector.

Hint

The softmax cancels the log and leaves p minus the one-hot target.

case 1

in: [[0,0],0]

expected: [-0.5,0.5]

case 2

in: [[1,2,3],2]

expected: [0.09003057317038046,0.24472847105479764,-0.3347590442251782]

case 3

in: [[0,0,0,0],1]

expected: [0.25,-0.75,0.25,0.25]

Starter codePython

def softmax_cross_entropy_logit_gradient(logits, label):
    # Your code here
    pass

3 test cases · run your code against them in the editor