MediumNLPnlp-039

TF-IDF for a Single Term

Problem

Return the TF-IDF weight of a single term in a single document: tf(token, doc) times log(N / df), where df is the number of documents in docs containing the term and N is len(docs).

Raw counts are used for tf unless log_tf is True, in which case tf is 1 + log(count). Return 0.0 when the term does not occur in doc or df is 0.

Hint

df counts documents, not occurrences; only the tf part uses the local count.

Test cases

case 1

in: ["cat",["the","cat","sat"],[["the","cat","sat"],["the","dog"]],false]

expected: 0.6931471805599453

case 2

in: ["the",["the","cat","the"],[["the","cat"],["the","dog"]],false]

expected: 0

case 3

in: ["dog",["cat"],[["cat"],["dog"]],false]

expected: 0

case 4

in: ["dog",["dog","dog"],[["cat"],["dog"],["bird"]],true]

expected: 1.860112299086919

Starter codePython

import math
def tfidf_single(token, doc, docs, log_tf=False):
    # Your code here
    pass

4 test cases · run your code against them in the editor