Logistic Regression Gradient

Problem

Compute the batch gradient of the average logistic log-loss with respect to weights w and intercept b.

With p_i = sigmoid(b + x_i . w), grad_b = mean(p_i - y_i) and grad_j = mean((p_i - y_i) * x_ij). Return [grad_b, grad_w]; empty X returns [0.0, [0.0] * len(w)].

Hint

The logistic log-loss gradient is the average of (sigmoid score - label) times each feature.

Test cases

case 1

in: [[[1],[2],[3]],[0,1,1],[0],0]

expected: [-0.16666666666666666,[-0.6666666666666666]]

case 2

in: [[[0],[1]],[1,1],[0],1]

expected: [-0.2689414213699951,[-0.13447071068499755]]

case 3

in: [[[1,0],[0,1]],[1,0],[0,0],0]

expected: [0,[-0.25,0.25]]

case 4

in: [[],[],[2,3],0]

expected: [0,[0,0]]

Starter codePython

import math
def logistic_gradient(X, y, w, b):
    # Returns [grad_b, grad_w]
    # Your code here
    pass

4 test cases · run your code against them in the editor