GLU Sigmoid Gate

Problem

Evaluate a gated linear unit on one input vector: out[j] = sigmoid(W[j] . x) * (V[j] . x), where sigmoid(a) = 1 / (1 + exp(-a)).

The signature is glu_forward(x, W, V). W and V are (hidden, d) matrices and x has length d; the result has length hidden.

Hint

A zero gate input still passes half of the value because sigmoid(0) is 0.5.

case 1

in: [[1],[[0]],[[5]]]

expected: [2.5]

case 2

in: [[1],[[1]],[[1]]]

expected: [0.7310585786]

case 3

in: [[0,1],[[1,0],[0,1]],[[1,1],[1,-1]]]

expected: [0.5,-0.7310585786]

case 4

in: [[1,1],[[-1,-1]],[[2,2]]]

expected: [0.4768116881]

Starter codePython

import math
def glu_forward(x, W, V):
    # Your code here
    pass

4 test cases · run your code against them in the editor