I had asked this question before here, but no response.
I want to concatenate newly labeled instances to the training set and retrain a model. I am using scikitlearn and the SVC classifier. In my approach, I first train a model on a small dataset, then I get some instances from a pool of unlabeled data based on least confidence strategy (the ones that are closer to the hyperplane) and then the selected instances become my new dataset which I need to preprocess. When I preprocess the text (I'm using tfidf as I am dealing with text) and the labels (label binarizer) I get a numpy array different in size from the numpy array used in the first training. How do I add the newly labeled data to training so that I can retrain the model with old data + new data?
My idea was to pad the sequences of the new preprocessed data to match the size of the old data and use this concatenated data to retrain the model, but I do not see changes in my results. So maybe my approach does not make sense. Can anyone help me with this?
#spliting the prepocessed dataset into train, test and pool
Xfeatures_train=Xfeatures[:300]
y_train_features = y_train[:300]
X_test=Xfeatures[300:400]
y_test=y_train[300:400]
X_pool=Xfeatures[400:]
y_pool=y_train[400:]
#function to train the model
def model(modelo, tipo):
svc= modelo
clf = tipo(svc)
clf.fit(Xfeatures_train,y_train_features)
clf_predictions = clf.predict(X_test)
clf_predictions_pool_prob=clf.decision_function(X_pool)
return clf_predictions_pool_prob
#training the model
preds_pool = model(LinearSVC(), OneVsRestClassifier)
#getting the indices from the instances in the pool
indices_list=df1.iloc[400:, 0].index.to_list()
#creating a dictionary that associating the indices (keys) with
#the model's predictions for the instances retrieved from the
#pool (values)
model1_probs_dic=dict(zip(indices_list,abs(preds_pool)))
#creating another dictionary that shows the indices of the
#instances whose model's predictions are close the decision
#boundary (hyperplane) with (distance < 0.1 from the hyperplane)
out_model1 = {
k: {str(i): x for i, x in enumerate(v, 1) if x < 0.1}
for k, v in model1_probs_dic.items()
}
print(out_model1)
#example output: {473: {'1': 0.03}, 474: {'2': 0.04}, 475: {},
476: {'7': 0.019146667060355727}, 477: {}}
#looping through the output dictionary to ignore the empty values
#and create two lists: one for the indices and the other for the
#values
dist=[]
idx=[]
for a,b in out_model1.items():
for i, j in b.items():
if b == None:
continue
else:
idx.append(a)
dist.append(j)
#creating a new dictionary with the indices and the distances
#below 0.1 and after this sorting the dictionary so I can get a
#dataset with the most uncertain instances to be added first inthe
#model for retraining
dic_idx_dist=dict(zip(idx, dist))
sorted_dic={k: v for k, v in sorted(dic_idx_dist.items(),
key=lambda item: item[1])}
#after sorting the dictionary, I can access the indices of
#the most uncertain instances in ascending order and use these
#indices to select from the pool
idx_DF=[]
for i in sorted_dic:
idx_DF.append(i)
#I select from my initial dataset these instances that I am
#assuming are the most informative to retrain the model. This is
#done based on the indices of those instances that are in the list
#idx_DF
uncertain_examples=df1.loc[idx_DF]
So, my new dataset is "uncertain_examples" which is a Data Frame with the instances (texts) and the labels. How to preprocess (convert into vectors using tfidf) and add these instances to my training set so that I can retrain the model using the old + the new? I do not know what would work well in this case. I would appreciate if anyone could help with this. I would also appreciate if someone could let me know if my overall approach could be improved to achieve my goal.