CLIP Similarity Matrix

Problem

Compute the normalized image-text similarity matrix in the CLIP style.

L2-normalize every embedding row, take the dot products of all image-text pairs and round to 4 decimal places.

case 1

in: [[[1,0],[0,1]],[[1,1],[1,-1]]]

expected: [[0.7071,0.7071],[0.7071,-0.7071]]

case 2

in: [[[3,4]],[[1,0],[0,1]]]

expected: [[0.6,0.8]]

case 3

in: [[[1,0]],[[0,1],[1,0]]]

expected: [[0,1]]

Starter codePython

def clip_similarity(image_embeds, text_embeds):
    # Your code here
    pass

3 test cases · run your code against them in the editor