MediumNLPnlp-181

Paraphrase Accuracy

Problem

Return the fraction of predictions that match the gold answer after canonicalization: lowercase, replace non-alphanumeric characters with spaces, and collapse whitespace.

Return 0.0 for an empty gold list, and zip so extra predictions are ignored.

Hint

Canonicalization removes punctuation and squeezes repeated spaces.

Test cases

case 1

in: [["The cat!","a dog"],["the cat","dog"]]

expected: 0.5

case 2

in: [["Hello"],["hello!"]]

expected: 1

case 3

in: [[],[]]

expected: 0

case 4

in: [["a b"],["a b"]]

expected: 1

case 5

in: [["x"],["y"]]

expected: 0

Starter codePython

import re
def paraphrase_accuracy(gold, predictions):
    # Your code here
    pass

5 test cases · run your code against them in the editor