CLIP Zero-Shot Logits

Problem

Compute zero-shot class probabilities for an image embedding.

Normalize the image and each per-class prompt embedding, scale the cosine similarities by temperature and apply softmax. Return the per-class probabilities rounded to 4 decimal places.

Test cases

case 1

in: [[1,0],[[1,0],[0,1],[-1,0]],1]

expected: [0.6652,0.2447,0.09]

case 2

in: [[1,0],[[0,1],[1,0]],2]

expected: [0.1192,0.8808]

case 3

in: [[1,0],[[0.6,0.8]],1]

expected: [1]

Starter codePython

def clip_zero_shot(image_embed, prompt_embeds, temperature):
    # Your code here
    pass

3 test cases · run your code against them in the editor