MediumNLPnlp-183

Hallucination Rate Proxy

Problem

Return the hallucination rate: the fraction of claims containing at least one token that is absent from the evidence tokens.

An empty claim is always supported, and an empty claim list gives 0.0.

Hint

all() over an empty claim is vacuously True, so empty claims are supported.

case 1

in: [[["the","cat"]],["the","cat"]]

expected: 0

case 2

in: [[["the","cat"],["the","dog"]],["the","cat"]]

expected: 0.5

case 3

in: [[["a"]],[]]

expected: 1

case 4

in: [[],["a"]]

expected: 0

case 5

in: [[[]],["a"]]

expected: 0

Starter codePython

def hallucination_rate_proxy(claims, evidence_tokens):
    # Your code here
    pass

5 test cases · run your code against them in the editor