MediumNLPnlp-252

Mean Reciprocal Rank for Retrieval

Problem

Compute mean reciprocal rank over queries. For each query take the 1-based rank of the first relevant document in its ranking, or 0.0 when no relevant document is retrieved, then average across queries.

rankings and relevant are parallel lists, one entry per query. Return 0.0 when there are no queries.

Hint

Reciprocal rank only cares about the first hit, not the ones after it.

Test cases

case 1

in: [[["a","b"],["x","y"]],[["b"],["z"]]]

expected: 0.25

case 2

in: [[],[]]

expected: 0

case 3

in: [[["a"],["b","c"]],[["a"],["c"]]]

expected: 0.75

case 4

in: [[["a","b","c"]],[["c"]]]

expected: 0.3333333333333333

case 5

in: [[["a"]],[[]]]

expected: 0

Starter codePython

def mean_reciprocal_rank(rankings, relevant):
    # Your code here
    pass

5 test cases · run your code against them in the editor