EasyNLPnlp-006

Punctuation-Stripping Tokenizer

Problem

Lowercase text, split on whitespace, then strip leading and trailing punctuation from each token using string.punctuation.

Tokens that become empty are dropped. Internal punctuation such as apostrophes and hyphens is preserved, so don't stays don't and well-known stays intact.

Hint

str.strip removes any characters from the given set at both ends only.

Test cases

case 1

in: ["Hello, World!"]

expected: ["hello","world"]

case 2

in: ["Don't stop - now!"]

expected: ["don't","stop","now"]

case 3

in: ["well-known fact"]

expected: ["well-known","fact"]

case 4

in: ["..."]

expected: []

case 5

in: [""]

expected: []

Starter codePython

import string
def strip_punctuation_tokenize(text):
    # Your code here
    pass

5 test cases · run your code against them in the editor