LLaVA Projector Params

Problem

Count the parameters of the two-layer MLP projector that maps vision features into the language model's embedding space: vision_dim*hidden + hidden + hidden*llm_dim + llm_dim.

The signature is llava_projector_params(vision_dim, llm_dim, hidden_dim). Return an integer.

Hint

The projector is tiny compared to the vision encoder and the language model.

Test cases

case 1

in: [1024,4096,4096]

expected: 20979712

case 2

in: [768,4096,4096]

expected: 19931136

case 3

in: [1024,1024,1024]

expected: 2099200

case 4

in: [1,1,1]

expected: 4

Starter codePython

def llava_projector_params(vision_dim, llm_dim, hidden_dim):
    # Your code here
    pass

4 test cases · run your code against them in the editor