ReLU Backward

Problem

Backpropagate through a ReLU. Given the upstream gradient dout and the pre-activation z, return dout[i] where z[i] > 0 and 0.0 otherwise.

The signature is relu_grad(dout, z).

Hint

The ReLU derivative is 1 for strictly positive pre-activations, 0 otherwise.

case 1

in: [[1,2,3],[-1,2,-3]]

expected: [0,2,0]

case 2

in: [[5],[0]]

expected: [0]

case 3

in: [[1,1],[0.5,0.25]]

expected: [1,1]

case 4

in: [[2,-2],[1e-9,-1e-9]]

expected: [2,0]

Starter codePython

def relu_grad(dout, z):
    # Your code here
    pass

4 test cases · run your code against them in the editor