All problems
Corpus Word Error Rate
Problem
Compute corpus-level WER by pooling edit operations. references and hypotheses are parallel lists of token lists.
Sum the Levenshtein distance over all pairs and divide by the total reference token count. Return 0.0 when all references are empty and all hypotheses are empty, and 1.0 when the references are empty but the hypotheses are not.
Hint
Aggregate edit distance and reference length across the whole corpus instead of averaging per-sentence rates.
Test cases
case 1
in: [[["a","b","c"],["a","b"]],[["a","c"],["a","c"]]]
expected: 0.4
case 2
in: [[["a"],[]],[["a"],[]]]
expected: 0
case 3
in: [[[],[]],[[],[]]]
expected: 0
case 4
in: [[[],[]],[["x"],["y"]]]
expected: 1
case 5
in: [[["a","b","c","d"]],[["a","b"]]]
expected: 0.5
Starter codePython
def corpus_wer(references, hypotheses):
# Your code here
pass5 test cases · run your code against them in the editor