MediumNLPnlp-261

Citation Attribution Score

Problem

Score how well the answer is attributed to its citations. For each cited passage compute the fraction of its distinct tokens that also occur in the answer, then return the maximum over citations.

Return 0.0 when there are no citations, or when citations exist but none of their tokens are in the answer.

Hint

Per-citation precision: the denominator is the citation's distinct token count.

Test cases

case 1

in: [["the","cat","sat"],[["cat","sat"],["dog","ran"]]]

expected: 1

case 2

in: [["the","cat"],[["cat","dog"],["the","cat","sat"]]]

expected: 0.6666666666666666

case 3

in: [[],[["a"]]]

expected: 0

case 4

in: [["a"],[]]

expected: 0

case 5

in: [["a","b"],[[]]]

expected: 0

Starter codePython

def citation_attribution_score(answer_tokens, citations):
    # Your code here
    pass

5 test cases · run your code against them in the editor