concatenate features from different steps in sklearn Pipeline

user_7205275 0
Viewed 298

I want to cascade 4 steps in a pipeline to build a supervised classifier:
(1) dimension reduction with PCA, obtaining a matrix_1 of s rows (samples) by c columns (components).

(2) feed the output matrix_1 from (1) to a KMeans blind separation, obtaining a matrix_2 of s rows (samples) by 1 column (group label).

(3) horizontally concatenate matrix_1 from (1) and matrix_2 from (2), obtaining a matrix_3 of s rows (samples) by c+1 columns (c components plus 1 label).

(4) feed the output matrix_3 from (3) to a neural network of MLPClassifier.

So my pipeline will look like this:

Pipeline(steps=[('step1', PCA()), ('step2', KMeans()), ('step3', myStep3(FastICA().components_, KMeans().labels_)), ('step4', MLPClassifier())])

Now my question is how to realized the 'step3' in the pipeline. Is there a sklearn function/class for me to replace "myStep3()"?

scikit-learn
2 Answers

One way to do this would be to duplicate your feature columns and apply a separate transformer to each. The first group would be getting the PCA transformation only, the second group would be getting the PCA and KMeans. This can be achieved using ColumnTransformer which assigns different transformations to different sets of columns. On a dummy example, it would look like this:

from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.cluster import KMeans
from sklearn.decomposition import PCA
from sklearn.neural_network import MLPClassifier

#Generate dummy data
df = pd.DataFrame({"a":np.random.random(100), "b": np.random.random(100), "y": np.random.choice(2,100)})

#Duplicate columns ["a", "b"] into ["a_new", "b_new"]
cols = ["a", "b"]
new_cols = [col_name + "_new" for col_name in cols]
df[new_cols] = df[cols]

#"cols" are receiving only the PCA
#"new_cols" are receiving a Pipeline made of PCA and KMeans
CT = ColumnTransformer([("onlyPCA", PCA(), cols),
                        ("PCA+KMeans", Pipeline([("PCA", PCA()), ("KMeans", KMeans())]), new_cols)])

#Wrap the whole thing into a Pipeline
pipe = Pipeline([("transformer", CT), ("classifier", MLPClassifier())])

pipe.fit(df[cols+new_cols], df.y)

Note that then you would also need to do duplicate the data in the predict step:

pipe.predict(df[cols+new_cols])
user_109566060

MaximeKan's answer is fantastic. I approached it in a different route using FeatureUnion, which can avoid the column duplication.

Pipeline(steps=[('ftrUn', FeatureUnion([('myDr', PCA()),('myDrKm', Pipeline([('myDr', PCA()),('myKM', KMeans())]))])),('myNN', MLPClassifier())])

The only issue here (also applies to the MaximeKan's above), is the output of KMeans().fit_transform(). This output are sample-center distance, rather than my original request of cluster labels.

user_72052750
Related