InfoNCE Mutual Information Bound

Problem

Return the InfoNCE contrastive loss in nats for one positive pair and the given negative similarities:

loss = -ln(exp(pos) / (exp(pos) + sum_i exp(neg_i)))

Hint

InfoNCE is a lower bound on the mutual information between the paired views.

case 1

in: [1,[0]]

expected: 0.3132616875182228

case 2

in: [0,[0,0]]

expected: 1.0986122886681098

case 3

in: [2,[0]]

expected: 0.12692801104297252

case 4

in: [1,[1,1]]

expected: 1.0986122886681098

Starter codePython

import math
def infonce_loss(pos_sim, neg_sims):
    # Your code here
    pass

4 test cases · run your code against them in the editor