All problems
BM25 Length Normalization Effect
Problem
Compare BM25 scoring with and without document-length normalization for a single term. With the given k1 and b the score is idf * tf * (k1 + 1) / (tf + k1 * (1 - b + b * doc_len / avg_doc_len)), and without normalization b is 0.
Return [with_b, without_b], or [0.0, 0.0] when avg_doc_len is not positive.
Hint
Only the denominator changes; idf and the numerator are shared.
Test cases
case 1
in: [3,100,50,2,1.5,0.75]
expected: [2.6666666666666665,3.3333333333333335]
case 2
in: [0,100,50,2,1.5,0.75]
expected: [0,0]
case 3
in: [3,100,50,2,1.5,0]
expected: [3.3333333333333335,3.3333333333333335]
case 4
in: [2,25,50,1,1.2,0.75]
expected: [1.6,1.375]
case 5
in: [2,25,0,1,1.2,0.75]
expected: [0,0]
Starter codePython
def bm25_length_normalization_effect(tf, doc_len, avg_doc_len, idf, k1, b):
# Your code here
pass5 test cases · run your code against them in the editor