MediumCalculusca-221

Log-Sum-Exp Hessian-Vector Product

Problem

The Hessian of the log-sum-exp function L(x) = log(sum exp(x_i)) is diag(s) - s s^T, where s is the softmax of x.

Compute the Hessian-vector product H v efficiently as s_i * (v_i - s dot v) for each i, and return the result vector.

Hint

The Hessian is positive semidefinite with the all-ones vector in its null space.

case 1

in: [[0,0],[1,0]]

expected: [0.25,-0.25]

case 2

in: [[1,2,3],[1,1,1]]

expected: [0,0,0]

case 3

in: [[0,1],[1,-1]]

expected: [0.3932238664829637,-0.3932238664829637]

Starter codePython

def logsumexp_hessian_vector(logits, v):
    # Your code here
    pass

3 test cases · run your code against them in the editor