EasyNLPnlp-255

Embedding Projection Cost

Problem

Return the parameter and compute cost of projecting n_vectors embeddings from orig_dim to new_dim with y = xW + b. Parameters are orig_dim * new_dim weights plus new_dim biases, and floating-point operations are n_vectors * orig_dim * new_dim multiplies plus n_vectors * new_dim additions.

Return [params, flops].

Hint

Both counts share the term new_dim; do not forget the bias additions.

Test cases

case 1

in: [1000,768,128]

expected: [98432,98432000]

case 2

in: [0,10,5]

expected: [55,0]

case 3

in: [10,4,0]

expected: [0,0]

case 4

in: [2,3,2]

expected: [8,16]

Starter codePython

def embedding_projection_cost(n_vectors, orig_dim, new_dim):
    # Your code here
    pass

4 test cases · run your code against them in the editor