MediumCalculusca-073

Gradient of Softmax Row

Problem

For the softmax vector s(x) with s_i = exp(x_i) / sum(exp(x_j)), the derivative of component i with respect to component j is s_i * (delta_ij - s_j).

Return row i of the Jacobian as a list over j, where delta_ij is 1 when i == j and 0 otherwise. Use max subtraction for stability.

Hint

Compute the full softmax first, then apply s_i * (delta - s_j) entrywise.

case 1

in: [[0,0],0]

expected: [0.25,-0.25]

case 2

in: [[0,1],1]

expected: [-0.19661193324148185,0.19661193324148185]

case 3

in: [[1,1,1],2]

expected: [-0.1111111111111111,-0.1111111111111111,0.22222222222222224]

case 4

in: [[2,-1],0]

expected: [0.045176659730912,-0.045176659730912144]

Starter codePython

def softmax_gradient_row(x, i):
    # Return [ds_i/dx_j for each j]
    # Your code here
    pass

4 test cases · run your code against them in the editor