Composing StandardScaler with a custom transformer

user_7648 0
Viewed 81

I have a custom transformer:

from sklearn.base import BaseEstimator, TransformerMixin
from datetime import datetime

class DayOfYearTransformer(BaseEstimator, TransformerMixin):
    def fit(self, X, y=None):
        return self  # nothing else to do
    def transform(self, X):
        date_array = X['date']
        day_of_year = [datetime.strptime(date_str, '%Y-%m-%d').date().timetuple().tm_yday for date_str in date_array]
        return np.c_[X.copy().drop(columns=['date']), day_of_year]

It converts a date string into the day of the year, i.e., a number between 1 and 365. Here is my pipeline:

from sklearn.compose import ColumnTransformer

num_attribs = []
cat_attribs = ['country', 'store', 'product']
date_attribs = ['date']

full_pipeline = ColumnTransformer([
    ("cat", OneHotEncoder(), cat_attribs),
    ("date", DayOfYearTransformer(), date_attribs),
])

merch_prepared = full_pipeline.fit_transform(merch)

I want to apply an sklearn.preprocessing.StandardScaler transform to the output of

    ("date", DayOfYearTransformer(), date_attribs),

How do I do this?

scikit-learn python
1 Answers

You can combine the two using a pipline that first does your DayOfYearTransformer followed by a StandardScaler. In code that reads like

from sklearn.pipeline import make_pipeline
full_pipeline = ColumnTransformer([
    ("cat", OneHotEncoder(), cat_attribs),
    ("date", make_pipeline(DayOfYearTransformer(), StandardScaler()), date_attribs),
])

where I have used the convenience function make_pipeline to build a sklearn Pipeline object.

user_140787580
Related