SigLIP Sigmoid Loss

Problem

Compute the SigLIP pairwise sigmoid loss. For each pair with logit z and label y (1 for matched, 0 otherwise), the target is +1 or -1 and the loss term is softplus(-target * (temperature * z + bias)). Return the mean over all pairs.

The signature is siglip_loss(logits, labels, temperature=1.0, bias=0.0).

Hint

SigLIP drops the global softmax, so it needs no all-gather of the full similarity matrix.

Test cases

case 1

in: [[[0]],[[1]],1,0]

expected: 0.6931471806

case 2

in: [[[0]],[[0]],1,0]

expected: 0.6931471806

case 3

in: [[[1,-1],[-1,1]],[[1,0],[0,1]],0.5,0]

expected: 0.4740769842

case 4

in: [[[0,0]],[[1,0]],1,0]

expected: 0.6931471806

Starter codePython

import math
def siglip_loss(logits, labels, temperature=1.0, bias=0.0):
    # Your code here
    pass

4 test cases · run your code against them in the editor