MediumNLPnlp-327

Digit Masking Tokenizer

Problem

Masking digits before tokenization reduces vocabulary sparsity: every digit is replaced by the same placeholder character.

Given text, return the masked text with each digit replaced by '#'.

Hint

Replace one character at a time.

case 1

in: ["user42"]

expected: "user##"

case 2

in: ["2024-01-02"]

expected: "####-##-##"

case 3

in: ["no digits"]

expected: "no digits"

Starter codePython

def digit_masking_tokenizer(text):
    # Your code here
    pass

3 test cases · run your code against them in the editor