LSTM Gates Forward

Problem

One LSTM cell step. W is a list of four weight matrices [Wf, Wi, Wg, Wo], each shaped (units, input_dim + units), and b is a list of four bias vectors [bf, bi, bg, bo]. Each matrix concatenates input weights and recurrent weights, so the pre-activation is W @ [x, h_prev] + b.

With f, i, o the sigmoids of the forget, input and output gates and g = tanh of the cell gate: c = f * c_prev + i * g and h = o * tanh(c). The signature is lstm_cell(x, h_prev, c_prev, W, b). Return [h, c].

Hint

The gate order is forget, input, cell, output; the cell state uses a running sum, not a dot product.

Test cases

case 1

in: [[1],[0],[0],[[[0,0]],[[0,0]],[[0,0]],[[0,0]]],[[0],[0],[0],[0]]]

expected: [[0],[0]]

case 2

in: [[1],[0],[1],[[[5,0]],[[0,0]],[[0,0]],[[0,0]]],[[0],[0],[0],[0]]]

expected: [[0.3793844859],[0.9933071491]]

case 3

in: [[2],[0],[0],[[[0,0]],[[0,0]],[[1,0]],[[0,0]]],[[0],[0],[0],[0]]]

expected: [[0.2239274687],[0.48201379]]

case 4

in: [[1,2],[0.5],[0.5],[[[0.5,0.5,0.1]],[[0.5,0.5,0.1]],[[0.5,0.5,0.1]],[[0.5,0.5,0.1]]],[[0.1],[0.1],[0.1],[0.1]]]

expected: [[0.6990010696],[1.198655871]]

Starter codePython

import math
def lstm_cell(x, h_prev, c_prev, W, b):
    # Your code here
    pass

4 test cases · run your code against them in the editor