MSE Backprop Through ReLU

Problem

Backpropagate mean squared error through a ReLU layer.

Forward: z = W @ x + b, y_pred = relu(z), L = (1/n) * sum((y_pred - y_true)^2) with n = len(y_pred). Return [dW, db] where the upstream gradient through ReLU is dz_i = 2 * (y_pred_i - y_true_i) / n if z_i > 0 else 0, dW[i][j] = dz_i * x_j, and db = dz.

Hint

The ReLU gate zeroes gradients for pre-activations at or below zero.

Test cases

case 1

in: [[1,0],[0],[[1,1]],[0]]

expected: [[[2,0]],[2]]

case 2

in: [[1,2],[1,1],[[1,0],[0,1]],[0,0]]

expected: [[[0,0],[1,2]],[0,1]]

case 3

in: [[1],[0],[[-1]],[-1]]

expected: [[[0]],[0]]

case 4

in: [[2,1],[3],[[0.5,0.5]],[0]]

expected: [[[-6,-3]],[-3]]

Starter codePython

def mse_backprop(x, y_true, W, b):
    # Return [dW, db]
    # Your code here
    pass

4 test cases · run your code against them in the editor