MediumNLPnlp-106

Span Extraction F1

Problem

Return the exact-match span F1 between predicted spans and gold spans, where each span is a [start, end] pair.

Duplicate spans are collapsed into sets, precision is matches / predicted spans, recall is matches / gold spans, and F1 = 2PR/(P+R). Return 1.0 when both sets are empty and 0.0 when either is empty or there are no matches.

Hint

Convert spans to tuples before putting them in a set.

Test cases

case 1

in: [[[0,2],[3,5]],[[0,2],[4,6]]]

expected: 0.5

case 2

in: [[[0,1]],[[2,3]]]

expected: 0

case 3

in: [[],[]]

expected: 1

case 4

in: [[[1,2]],[]]

expected: 0

case 5

in: [[[0,2]],[[0,2]]]

expected: 1

Starter codePython

def span_extraction_f1(predicted, gold):
    # Your code here
    pass

5 test cases · run your code against them in the editor