ReLU Backward Subgradient

Problem

The ReLU activation is max(0, x) and its subgradient is 1 for x > 0 and 0 for x <= 0.

Given the upstream gradient vector grad and the pre-activation vector x, return the elementwise product of grad with this subgradient, choosing 0 at x == 0.

Hint

The ReLU mask zeroes the gradient wherever the input is not strictly positive.

Test cases

case 1

in: [[1,-2,3],[2,-1,0]]

expected: [1,0,0]

case 2

in: [[2,2,2],[0,0,0]]

expected: [0,0,0]

case 3

in: [[-1,5],[-3,2]]

expected: [0,5]

case 4

in: [[1],[0.5]]

expected: [1]

Starter codePython

def relu_backward(grad, x):
    # Your code here
    pass

4 test cases · run your code against them in the editor