HardNLPnlp-283

Entity-Level Precision and Recall

Problem

Compute per-label entity precision, recall, and F1. gold and pred are lists of [start, end, label] spans, matched exactly on start and end within each label with multiplicity clipping.

Return a dictionary mapping every label that appears in either list to [precision, recall, f1]. A zero denominator gives 0.0 for that quantity.

Hint

Group counts by label first, then clip match counts per (start, end) key.

Test cases

case 1

in: [[[0,2,"PER"],[3,4,"PER"],[0,2,"LOC"]],[[0,2,"PER"],[3,4,"PER"],[0,3,"LOC"]]]

expected: {"LOC":[0,0,0],"PER":[1,1,1]}

case 2

in: [[],[]]

expected: {}

case 3

in: [[[0,1,"A"]],[[0,1,"B"]]]

expected: {"A":[0,0,0],"B":[0,0,0]}

case 4

in: [[[0,1,"A"],[0,1,"A"]],[[0,1,"A"]]]

expected: {"A":[1,0.5,0.6666666666666666]}

case 5

in: [[[0,1,"X"],[1,2,"X"]],[[0,1,"X"],[1,2,"Y"]]]

expected: {"X":[1,0.5,0.6666666666666666],"Y":[0,0,0]}

Starter codePython

def entity_level_prf(gold, pred):
    # Your code here
    pass

5 test cases · run your code against them in the editor