MediumNLPnlp-284

Chunking F1 from BIO Tags

Problem

Compute chunk-level micro-F1 directly from BIO tag sequences. Decode both sequences into [start, end, label] chunks using IOB2 rules, where end is exclusive and an I- tag may start a chunk when no chunk is open or the label changes.

Exact chunk matches are clipped by multiplicity, and precision, recall, and F1 are computed over the pooled counts. Return 1.0 when both sequences produce no chunks and 0.0 when there are no matches.

Hint

Decode both tag sequences to chunk triples, then run clipped micro-F1 over the triples.

Test cases

case 1

in: [["B-NP","I-NP","O","B-VP"],["B-NP","I-NP","O","B-VP"]]

expected: 1

case 2

in: [["B-NP","I-NP","O"],["O","B-NP","I-NP"]]

expected: 0

case 3

in: [[],[]]

expected: 1

case 4

in: [["B-NP","I-NP","O","B-VP"],["B-NP","I-NP","B-VP","O"]]

expected: 0.5

case 5

in: [["O","B-PER","I-PER"],["O","B-LOC","I-PER"]]

expected: 0

Starter codePython

def chunking_f1(gold_tags, pred_tags):
    # Your code here
    pass

5 test cases · run your code against them in the editor