Model Top-1 Agreement

Problem

Measure how often two models agree on their top-1 prediction: the fraction of rows where the argmax of logits_a equals the argmax of logits_b. Break ties by the smaller index.

The signature is top1_agreement(logits_a, logits_b). Both inputs are 2D logit matrices with the same shape.

Hint

Agreement is a useful cheap ensemble diversity signal before computing full accuracy.

Test cases

case 1

in: [[[1,2,3],[3,2,1]],[[1,2,3],[3,2,1]]]

expected: 1

case 2

in: [[[1,2,3],[3,2,1]],[[3,2,1],[1,2,3]]]

expected: 0

case 3

in: [[[1,1]],[[1,2]]]

expected: 0

case 4

in: [[[0,0]],[[5,5]]]

expected: 1

Starter codePython

def top1_agreement(logits_a, logits_b):
    # Your code here
    pass

4 test cases · run your code against them in the editor