I want to classify data into categories basis the information present in 2 different columns. I want to apply TFIDF vectorisation to the two columns independently - i.e. have separate vectors for the information present in the 2 columns and transform the 2 test data columns separately as well.
This is the code that I am using for vectorising only one column (VLOB_D&B) -
Train_X, Test_X, Train_Y, Test_Y =
model_selection.train_test_split(Corpus['VLOB_D&B'],Corpus['category_id'],test_size=0.3, stratify =
Corpus['category_id'])
Tfidf_vect = TfidfVectorizer(sublinear_tf=True, min_df=2, ngram_range=(1, 2))
Tfidf_vect.fit(Train_X)
Train_X_Tfidf = Tfidf_vect.transform(Train_X)
Test_X_Tfidf = Tfidf_vect.transform(Test_X)
I do not want to concatenate the information as the two columns hold different significance. How can i apply TFIDF vectorisation independently on separate columns?