MediumNLPnlp-164

Persona Consistency Score

Problem

Score persona consistency: for each persona sentence, take the maximum Jaccard similarity between its token set and any response sentence token set, then average over persona sentences.

Return 0.0 when there are no persona sentences, and an empty response set contributes 0.0 for each persona sentence.

Hint

Jaccard compares token sets, and each persona sentence picks its best matching response.

Test cases

case 1

in: [[["i","love","coffee"]],[["i","love","coffee"]]]

expected: 1

case 2

in: [[["a","b"]],[["c"]]]

expected: 0

case 3

in: [[["a","b"],["c"]],[["a"],["c","d"]]]

expected: 0.5

case 4

in: [[],[["x"]]]

expected: 0

case 5

in: [[["x"]],[]]

expected: 0

Starter codePython

def persona_consistency_score(persona_sentences, response_sentences):
    # Your code here
    pass

5 test cases · run your code against them in the editor