Gradient of Log-Sum-Exp

Problem

Compute the gradient of the log-sum-exp function L(x) = log(sum(exp(x_i))) at a point x.

The gradient is the softmax vector: grad_i = exp(x_i) / sum(exp(x_j)). Subtract the maximum entry before exponentiating for numerical stability; the result is unchanged.

Hint

The gradient of log-sum-exp is the softmax of the inputs.

Test cases

case 1

in: [[0,0]]

expected: [0.5,0.5]

case 2

in: [[0,1]]

expected: [0.2689414213699951,0.7310585786300049]

case 3

in: [[-1,1]]

expected: [0.11920292202211755,0.8807970779778823]

case 4

in: [[2,2,2]]

expected: [0.3333333333333333,0.3333333333333333,0.3333333333333333]

Starter codePython

def lse_gradient(x):
    # Return the gradient vector
    # Your code here
    pass

4 test cases · run your code against them in the editor