MediumNLPnlp-244

BM25 Multi-Term Query Score

Problem

Score a multi-term query under BM25. Sum the contribution of each distinct query term with a positive term frequency: idf * tf * (k1 + 1) / (tf + k1 * (1 - b + b * doc_len / avg_doc_len)), with idf = log(1 + (n_docs - df + 0.5) / (df + 0.5)).

Return 0.0 when n_docs or avg_doc_len is not positive.

Hint

Log is the natural log, and duplicate query terms count once.

Test cases

case 1

in: [["cat","dog"],{"cat":2,"dog":0},{"cat":10,"dog":5},50,50,100]

expected: 3.233921799539688

case 2

in: [["a"],{"a":1},{"a":1},20,40,4]

expected: 1.5535132959044335

case 3

in: [[],{"a":1},{"a":1},20,40,4]

expected: 0

case 4

in: [["a","b","a"],{"a":3,"b":1},{"a":2,"b":8},100,50,20]

expected: 6.299047496674246

case 5

in: [["a"],{"a":2},{"a":1},25,50,10,0,0]

expected: 1.992430164690206

Starter codePython

import math
def bm25_query_score(query_terms, tf, df, doc_len, avg_doc_len, n_docs, k1=1.5, b=0.75):
    # Your code here
    pass

5 test cases · run your code against them in the editor