Link Prediction AUC Lite

Problem

Evaluate link prediction scores with a simplified AUC. positives and negatives are lists of held-out edge pairs, and scores is a dictionary mapping pair keys like 0,2 (endpoints sorted, comma separated) to a score.

For every positive-negative combination award 1 when the positive scores higher, 0.5 on a tie, and 0 otherwise. Return the average over all combinations, or 0.0 when a set is empty.

Hint

AUC is the probability that a random positive edge outscores a random negative one.

Test cases

case 1

in: [[[0,2],[1,3]],[[0,1],[2,3]],{"0,2":0.9,"1,3":0.8,"0,1":0.1,"2,3":0.7}]

expected: 1

case 2

in: [[[0,2]],[[0,1]],{"0,2":0.4,"0,1":0.4}]

expected: 0.5

case 3

in: [[[0,2]],[[0,1]],{"0,2":0.2,"0,1":0.9}]

expected: 0

Starter codePython

def link_prediction_auc(positives, negatives, scores):
    # Your code here
    pass

3 test cases · run your code against them in the editor