I am looking for a class or method that takes a long string of many 100s of words and tokenizes, removes the stop words and stems for use in an IR system.
For example:
"The big fat cat, said 'your funniest guy i know' to the kangaroo..."
the tokenizer would remove the punctuation and return an ArrayList of words
the stop word remover would remove words like "the", "to", etc
the stemmer would reduce each word the their 'root', for example 'funniest' would become funny
Many thanks in advance.