MediumCalculusca-280

GELU Tanh Derivative Vector

Problem

The tanh approximation of GELU is f(z) = 0.5 * z * (1 + tanh(u)) with u = sqrt(2/pi) * (z + 0.044715 * z^3).

Its derivative is 0.5 * (1 + tanh(u)) + 0.5 * z * (1 - tanh(u)^2) * du/dz where du/dz = sqrt(2/pi) * (1 + 3 * 0.044715 * z^2).

Given the values, return the derivative list.

Hint

Differentiate tanh(u) as (1 - tanh(u)^2) * du/dz and apply the product rule.

case 1

in: [[0,1,-1]]

expected: [0.5,1.0829640838457826,-0.08296408384578258]

case 2

in: [[0.5,2]]

expected: [0.8673699035346424,1.0860992566236183]

case 3

in: [[-3,3]]

expected: [-0.011584166630969516,1.0115841666309695]

Starter codePython

def gelu_tanh_derivative(values):
    # Your code here
    pass

3 test cases · run your code against them in the editor