RNN Forward One Step

Problem

One step of a vanilla RNN: h = tanh(Wx @ x + Wh @ h_prev + b).

The signature is rnn_step(x, h_prev, Wx, Wh, b). Wx has shape (hidden, input_dim), Wh has shape (hidden, hidden) and b has length hidden.

Hint

Compute the affine pre-activation for every hidden unit, then apply tanh.

case 1

in: [[1],[0],[[0.5]],[[0.5]],[0]]

expected: [0.4621171573]

case 2

in: [[0,0],[0,0],[[1,0],[0,1]],[[0,0],[0,0]],[0,0]]

expected: [0,0]

case 3

in: [[1,2],[0.5,-0.5],[[1,0],[0,1]],[[0.1,0],[0,0.1]],[0,0]]

expected: [0.7818063576,0.9603193885]

case 4

in: [[],[1],[[]],[[0.5]],[0]]

expected: [0.4621171573]

Starter codePython

import math
def rnn_step(x, h_prev, Wx, Wh, b):
    # Your code here
    pass

4 test cases · run your code against them in the editor