MediumTime Seriests-242

Transformer Context Window

Problem

Estimate the effective context of stacked strided attention layers: 1 + (window - 1) * (1 + stride * (layers - 1)).

Return the effective context length in positions. Each extra layer expands the reach of a token.

Hint

Even a single attentive layer with window w already spans w positions.

case 1

in: [2,3,1]

expected: 5

case 2

in: [3,3,2]

expected: 11

case 3

in: [1,5,2]

expected: 5

case 4

in: [0,1,2]

expected: 1

Starter codePython

def transformer_context_window(layers, window, stride):
    # Your code here
    pass

4 test cases · run your code against them in the editor