add new instances selected from a pool to a training set and retrain with old + new instances

Viewed 11

I had asked this question before here, but no response.

I want to concatenate newly labeled instances to the training set and retrain a model. I am using scikitlearn and the SVC classifier. In my approach, I first train a model on a small dataset, then I get some instances from a pool of unlabeled data based on least confidence strategy (the ones that are closer to the hyperplane) and then the selected instances become my new dataset which I need to preprocess. When I preprocess the text (I'm using tfidf as I am dealing with text) and the labels (label binarizer) I get a numpy array different in size from the numpy array used in the first training. How do I add the newly labeled data to training so that I can retrain the model with old data + new data?

My idea was to pad the sequences of the new preprocessed data to match the size of the old data and use this concatenated data to retrain the model, but I do not see changes in my results. So maybe my approach does not make sense. Can anyone help me with this?

#spliting the prepocessed dataset into train, test and pool 
Xfeatures_train=Xfeatures[:300]
y_train_features = y_train[:300]
X_test=Xfeatures[300:400]
y_test=y_train[300:400]
X_pool=Xfeatures[400:]
y_pool=y_train[400:]

#function to train the model

 def model(modelo, tipo):
    svc= modelo
    clf = tipo(svc)
    clf.fit(Xfeatures_train,y_train_features)
    clf_predictions = clf.predict(X_test)
    clf_predictions_pool_prob=clf.decision_function(X_pool)
    return clf_predictions_pool_prob 

#training the model
    
preds_pool = model(LinearSVC(), OneVsRestClassifier)

#getting the indices from the instances in the pool 
 
  indices_list=df1.iloc[400:, 0].index.to_list()

 #creating a dictionary that associating the indices (keys) with 
 #the model's predictions for the instances retrieved from the 
 #pool (values)

model1_probs_dic=dict(zip(indices_list,abs(preds_pool)))


#creating another dictionary that shows the indices of the 
#instances whose model's predictions are close the decision 
#boundary (hyperplane) with (distance < 0.1 from the hyperplane)

out_model1 = {
    k: {str(i): x for i, x in enumerate(v, 1) if x < 0.1}
    for k, v in model1_probs_dic.items()
}
print(out_model1)

#example output: {473: {'1': 0.03}, 474: {'2': 0.04}, 475: {}, 
        476: {'7': 0.019146667060355727}, 477: {}}

#looping through the output dictionary to ignore the empty values 
#and create two lists: one for the indices and the other for the 
#values

dist=[]
idx=[]
for a,b in out_model1.items():
  for i, j in b.items():
    if b == None:
      continue
    else:
      idx.append(a)
      dist.append(j)
#creating a new dictionary with the indices and the distances 
#below 0.1 and after this sorting the dictionary so I can get a 
#dataset with the most uncertain instances to be added first inthe 
#model for retraining


 dic_idx_dist=dict(zip(idx, dist))

sorted_dic={k: v for k, v in sorted(dic_idx_dist.items(), 
key=lambda item: item[1])}

#after sorting the dictionary, I can access the indices of 
#the most uncertain instances in ascending order and use these 
#indices to select from the pool  

idx_DF=[]
for i in sorted_dic:
  idx_DF.append(i)

#I select  from my initial dataset these instances that I am 
#assuming are the most informative to retrain the model. This is 
#done based on the indices of those instances that are in the list   
#idx_DF

uncertain_examples=df1.loc[idx_DF]

So, my new dataset is "uncertain_examples" which is a Data Frame with the instances (texts) and the labels. How to preprocess (convert into vectors using tfidf) and add these instances to my training set so that I can retrain the model using the old + the new? I do not know what would work well in this case. I would appreciate if anyone could help with this. I would also appreciate if someone could let me know if my overall approach could be improved to achieve my goal.

0 Answers
Related