HardNLPnlp-189

METEOR-lite

Problem

Return a lite METEOR score. Match hypothesis tokens against a reference multiset in order, counting matches and the number of maximal contiguous matched chunks.

With P = matches / len(hypothesis), R = matches / len(reference), Fmean = 10PR/(R + 9P), and penalty = 0.5 * (chunks / matches) ** 3, return Fmean * (1 - penalty). Return 0.0 when there are no matches.

Hint

The fragmentation penalty grows with the cube of chunks relative to matches.

Test cases

case 1

in: [["the","cat","sat"],["the","cat","sat"]]

expected: 0.9814814814814815

case 2

in: [["the","cat"],["the","dog"]]

expected: 0.25

case 3

in: [["cat"],["cat","dog"]]

expected: 0.2631578947368421

case 4

in: [[],[]]

expected: 0

case 5

in: [["a"],[]]

expected: 0

Starter codePython

def meteor_lite(hypothesis, reference):
    # Your code here
    pass

5 test cases · run your code against them in the editor