I am working on a machine learning regression task with mixed continuous and categorical features in Python .
I apply one-hot encoding on categorical features as can be seen below:
from sklearn.datasets import fetch_openml
from sklearn.model_selection import train_test_split
from sklearn.impute import SimpleImputer
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, MinMaxScaler
# -----------------------------------------------------------------------------
# Data
# -----------------------------------------------------------------------------
# Ames
X, y = fetch_openml(name="house_prices", as_frame=True, return_X_y=True)
# In this dataset, categorical features have "object" or "non-numerical" data-type.
numerical_features = X.select_dtypes(include='number').columns.tolist() # 37
categorical_features = X.select_dtypes(include='object').columns.tolist() # 43
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.3, random_state=0)
# -----------------------------------------------------------------------------
# Data preprocessing
# -----------------------------------------------------------------------------
numerical_preprocessor = Pipeline(steps=[
('impute', SimpleImputer(strategy='mean')),
('scale', MinMaxScaler())
])
categorical_preprocessor = Pipeline(steps=[
('impute', SimpleImputer(strategy='most_frequent')),
('one-hot', OneHotEncoder(handle_unknown='ignore', sparse=False))
])
preprocessor = ColumnTransformer(transformers=[
('number', numerical_preprocessor, numerical_features),
('category', categorical_preprocessor, categorical_features)
],
verbose_feature_names_out=True,
)
X_train_processed = preprocessor.fit_transform(X_train)
X_test_processed = preprocessor.transform(X_test)
I want to remove highly correlated features by the following algorithm:
- Find Pearson correlation coefficient between all features.
- If correlation > threshold:
- Drop one of the features which has lower correlation with objective variable (which is a continuous variable)
However, I am not sure which method is suitable to calculate correlation between :
- continuous features & one-hot encoded categorical features
- one-hot encoded categorical features & continuous objective variable
Any advice is appreciated.
Assume that the machine learning task is a classification task. Which method do you recommend to calculate correlation between :
- one-hot encoded categorical features & categorical objective variable
- continuous features & categorical objective variable