LSTM Backward Gate Gradient

Problem

Backward pass through one LSTM unit given the upstream gradients dh and dc_next, the gate activations f, i, g, o and the cell states c_prev and c. Compute dc = dh * o * (1 - tanh(c)^2) + dc_next, then df = dc * c_prev * f * (1 - f), di = dc * g * i * (1 - i), dg = dc * i * (1 - g^2), do = dh * tanh(c) * o * (1 - o).

The signature is lstm_gate_gradients(dh, dc_next, f, i, g, o, c_prev, c). All values are scalars. Return [df, di, dg, do, dc].

Hint

The output gate gradient uses the old cell state's tanh, while the other gates use dc.

Test cases

case 1

in: [1,0,0.5,0.5,0.5,0.5,0,0.25]

expected: [0,0.0587509281,0.1762527842,0.0612296656,0.4700074244]

case 2

in: [2,0.5,0.8,0.4,0.6,0.7,1,1.2]

expected: [0.1483244792,0.1334920312,0.2373191666,0.3501349349,0.9270279947]

case 3

in: [-1,0,0.2,0.9,0.3,0.5,0.5,0.4]

expected: [-0.0342255514,-0.0115511236,-0.3503840829,-0.0949872406,-0.427819393]

case 4

in: [1,1,0.6,0.6,0.6,0.6,0,0]

expected: [0,0.2304,0.6144,0,1.6]

Starter codePython

import math
def lstm_gate_gradients(dh, dc_next, f, i, g, o, c_prev, c):
    # Your code here
    pass

4 test cases · run your code against them in the editor