MediumNLPnlp-105

Token-Level F1

Problem

Return the token-level F1 between a predicted token list and a gold token list, using clipped multiset matching: matches = sum over tokens of min(count in prediction, count in gold).

Precision is matches / len(predicted), recall is matches / len(gold), and F1 = 2PR/(P+R). Return 1.0 when both lists are empty and 0.0 when there are no matches.

Hint

Clipping prevents repeated predicted tokens from inflating recall.

Test cases

case 1

in: [["a","b","c"],["a","b","c"]]

expected: 1

case 2

in: [["a","b"],["a","c"]]

expected: 0.5

case 3

in: [["a","a","b"],["a","b","b"]]

expected: 0.6666666666666666

case 4

in: [[],[]]

expected: 1

case 5

in: [["x"],["y"]]

expected: 0

Starter codePython

def token_level_f1(predicted, gold):
    # Your code here
    pass

5 test cases · run your code against them in the editor