MediumNLPnlp-299

Sentence Edit Distance and WER

Problem

Compute both the token-level Levenshtein distance and the word error rate between a reference and a hypothesis. reference and hypothesis are lists of tokens.

Return [edit_distance, wer], where edit_distance counts substitutions, insertions, and deletions, and wer is edit_distance divided by the reference length. An empty reference gives 0.0 for an empty hypothesis and 1.0 otherwise.

Hint

Fill the standard edit-distance table once, then derive the rate from the final cell.

Test cases

case 1

in: [["a","b","c"],["a","b","c"]]

expected: [0,0]

case 2

in: [["a","b","c"],["a","c"]]

expected: [1,0.3333333333333333]

case 3

in: [["a","b"],["b","a"]]

expected: [2,1]

case 4

in: [["a"],["a","b"]]

expected: [1,1]

case 5

in: [[],["x"]]

expected: [1,1]

Starter codePython

def sentence_edit_distance_wer(reference, hypothesis):
    # Your code here
    pass

5 test cases · run your code against them in the editor