Backprop Gradient

Problem

Compute gradients of MSE loss (L = 0.5 * sum((y_pred - y_true)^2)) for a 2-layer MLP with ReLU hidden activation and linear output.

Forward: z1 = W1 @ x + b1 a1 = relu(z1) z2 = W2 @ a1 + b2 = y_pred

Backward (let dz2 = y_pred - y_true): gW2[i][j] = dz2[i] * a1[j] gb2[i] = dz2[i] da1[j] = sum_i(W2[i][j] * dz2[i]) dz1[j] = da1[j] if z1[j] > 0 else 0 gW1[i][j] = dz1[i] * x[j] gb1[i] = dz1[i]

Return [gW2, gb2, gW1, gb1].

Hint

Work backwards from dz2 = y_pred - y_true. Multiply by local Jacobians at each layer.

Test cases

case 1

in: [[1],[1],[[0.5],[-0.5]],[0,0],[[1,1]],[0]]

expected: [[[-0.25,0]],[-0.5],[[-0.5],[0]],[-0.5,0]]

case 2

in: [[1,2],[5],[[1,0],[0,1]],[0,0],[[1,1]],[0]]

expected: [[[-2,-4]],[-2],[[-2,-4],[-2,-4]],[-2,-2]]

case 3

in: [[2],[3],[[1]],[1],[[2]],[0.5]]

expected: [[[10.5]],[3.5],[[14]],[7]]

Starter codePython

def backprop_gradient(x, y_true, W1, b1, W2, b2):
    # Returns [gW2, gb2, gW1, gb1]
    # Your code here
    pass

3 test cases · run your code against them in the editor