GRU Update Gate Gradient

Problem

Backward pass for the update-gate path of one scalar GRU unit. With h = (1 - z) * h_prev + z * n and candidate pre-activation gradient dpre_n, let dsh = dpre_n * uh be the gradient through (r * h_prev) where uh is the candidate recurrent weight. Then dz = dh * (n - h_prev), dr = dsh * h_prev and dh_prev = dh * (1 - z) + dsh * r.

The signature is gru_update_gate_gradient(dh, h_prev, z, n, r, dpre_n, uh). All values are scalars. Return [dz, dr, dh_prev].

Hint

The update gate trades off the old hidden state and the candidate; the reset gate scales the recurrent term.

Test cases

case 1

in: [1,0.5,0.3,0.8,0.6,0.5,2]

expected: [0.3,0.5,1.3]

case 2

in: [2,-1,0.5,0,0.2,1,0.5]

expected: [2,-0.5,1.1]

case 3

in: [-1,0,0.9,0.5,0.5,0.5,1]

expected: [-0.5,0,0.15]

case 4

in: [0.5,2,0.1,0.9,0.9,2,0]

expected: [-0.55,0,0.45]

Starter codePython

def gru_update_gate_gradient(dh, h_prev, z, n, r, dpre_n, uh):
    # Your code here
    pass

4 test cases · run your code against them in the editor