EasyNLPnlp-072

Attention Scaling Factor

Problem

Return the scaled dot-product attention scaling factor 1 / sqrt(d_k).

d_k is the key dimension and is assumed positive.

Hint

The factor grows smaller as the key dimension grows.

case 1

in: [1]

expected: 1

case 2

in: [4]

expected: 0.5

case 3

in: [2]

expected: 0.7071067811865475

case 4

in: [64]

expected: 0.125

case 5

in: [16]

expected: 0.25

Starter codePython

import math
def attention_scaling_factor(d_k):
    # Your code here
    pass

5 test cases · run your code against them in the editor