GRU Gates Forward

Problem

One GRU cell step with reset and update gates. W = [Wz, Wr, Wh] and U = [Uz, Ur, Uh] hold the input and recurrent weight matrices, and b = [bz, br, bh] the biases.

Compute z = sigmoid(Wz x + Uz h + bz), r = sigmoid(Wr x + Ur h + br), n = tanh(Wh x + Uh (r * h) + bh) and h_new = (1 - z) * h + z * n, where * is element-wise. The signature is gru_cell(x, h_prev, W, U, b). Return the new hidden vector.

Hint

The reset gate scales the previous hidden state only inside the candidate pre-activation.

Test cases

case 1

in: [[1],[1],[[[0,0]],[[0,0]],[[0,0]]],[[[0]],[[0]],[[0]]],[[0],[0],[0]]]

expected: [0.5]

case 2

in: [[1],[0],[[[10,0]],[[0,0]],[[1,0]]],[[[0]],[[0]],[[0]]],[[0],[0],[0]]]

expected: [0.7615595812]

case 3

in: [[1],[1],[[[0,0]],[[10,0]],[[1,0]]],[[[0]],[[0]],[[0]]],[[0],[0],[0]]]

expected: [0.880797078]

case 4

in: [[0.5,-0.5],[0.2],[[[0.1,0.2,0.3]],[[0.1,0.2,0.3]],[[0.1,0.2,0.3]]],[[[0.5]],[[0.5]],[[0.5]]],[[0],[0],[0]]]

expected: [0.0981410087]

Starter codePython

import math
def gru_cell(x, h_prev, W, U, b):
    # Your code here
    pass

4 test cases · run your code against them in the editor