How to prepare comment threads for text classification

Viewed 13

Larger Goal:

Classify comment threads as spam or not spam.

Specific Goal:

Prepare comment threads for text classification

Example of a spam comment:

Joey Dyson: "I am so glad I was able to make $9000 in the last 30 days with Mrs Felicitas Harrison"
|-> Caron Jackson: "Can you tell me how you did this?"
|-> aaron1: "I'm also interested"
|-> John Ellison: "I'm also interested"
|-> Lucy Carlson: "Mrs Harrison can be contacted on +44 123..."
|-> Lucy Carlson: "456 789"

There is only one layer of comments. Put another way, a child comment cannot have another child comment.

How I plan to use the prepared comment threads

x = np.array(data["PREPARED_COMMENT_THREAD"])

cv = CountVectorizer()
x = cv.fit_transform(x)
xtrain, xtest, ytrain, ytest = train_test_split(x, y, 
                                                test_size=0.2, 
                                                random_state=42)

model.fit(xtrain, ytrain)
print(model.score(xtest, ytest))
0 Answers
Related