I am running a code to do a binary classification and predict labels afterwards. The code runs perfectly with one specific database of size 257673 rows and 47 columns. When I try with one of 91690 rows and 10 columns, which is certainly smaller, I get the error call:
TerminatedWorkerError: A worker process managed by the executor was unexpectedly terminated. This could be caused by a segmentation fault while calling the function or by an excessive memory usage causing the Operating System to kill the worker. The exit codes of the workers are {SIGKILL(-9)}
I am using a cross validation line with n_job=-1
cross_val_score(model, X.drop(target,axis=1), X[target], cv=outer_cv, n_jobs=-1, scoring='neg_mean_squared_error')
outer_cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=1)
model is any of the sklearn algorithms, I have tried with AdaBoostClassifier, LogisticRegression, KNN, SVM, GradientBoosting, RandomForest, DecisionTreeClassifier... and many others and I keep getting the same error.
I have tried changing n_jobs=-2, 1, 2 and still the error persists. I am running the code on a jupyter notebook and my laptop has the following properties:
Ubuntu 18.04.4 LTS
RAM: 15,5 Gb
Processor: Intel® Core™ i7-8550U CPU @ 1.80GHz × 8
How could I solve this issue?