Cannot concatenate object of type '<class 'numpy.ndarray'>'

Gianluca Coletta 0
Viewed 30

i have a problem when i try to concatenate train set and validation set. I split my dataset into train set, validation set and test set. Then i scale them with 'StandardScaler()':

X_train, X_test, t_train, t_test = train_test_split(x, t, test_size=0.20, random_state=1)
X_train, X_valid, t_train, t_valid = train_test_split(X_train, t_train, test_size=0.25, random_state=1)
sc = StandardScaler()
X_train = sc.fit_transform(X_train)
X_valid = sc.transform(X_valid)
X_test = sc.transform(X_test)

Then after model selection i want concatenate training and validation set:

X_train = pd.concat([X_train, X_valid])
t_train = pd.concat([t_train, t_valid])

But it doesn't work. I give me that error:

cannot concatenate object of type '<class 'numpy.ndarray'>'; only Series and DataFrame objs are valid

Can someone help me? Thanks

pandas numpy python
1 Answers

X_train, X_valid, t_train, t_valid are all numpy arrays so they need to be concatenated using numpy:

X_train = np.concatenate([X_train, X_valid])
t_train = np.concatenate([t_train, t_valid])

As suggested in the comments it is most likely not a good idea to merge train and validation sets together. Make sure you understand why datasets are split into training, testing and validation parts. You can apply cross validation to use all the data for train/test/valid in multiple steps.

Michal Racko0
Related