Attention Dropout (Seeded)

Problem

Apply inverted dropout to an attention weight matrix. Call random.seed(seed), draw one random number per matrix entry in row-major order, zero entries with probability p and scale the rest by 1 / (1 - p).

The signature is attention_dropout(weights, p, seed).

Hint

Dropout on attention discourages a token from relying on a single source position.

Test cases

case 1

in: [[[0.5,0.5]],0.5,1]

expected: [[0,1]]

case 2

in: [[[0.2,0.8],[0.6,0.4]],0.25,3]

expected: [[0,1.0666666667],[0.8,0.5333333333]]

case 3

in: [[[1]],0,9]

expected: [[1]]

case 4

in: [[[0.5,0.5],[0.5,0.5]],0.75,11]

expected: [[0,0],[2,0]]

Starter codePython

import random
def attention_dropout(weights, p, seed):
    # Your code here
    pass

4 test cases · run your code against them in the editor