Sklearn transformer to ensure input variable length is consistent?

user_17582153 0
Viewed 33

I have a sklearn pipeline that has a step which fractionally differentiates the features to ensure stationarity. If a feature needs to be made stationary - its length is reduced by 1. If it does not require fractional differentiation, its length remains the same (I think).

from fracdiff.sklearn import Fracdiff
...
self.clf = Pipeline(
                steps=[("preprocessor", self.preprocessor),
                       ("simple_imputer", simple_imputer),
                       ("fracdiff", Fracdiff()),
                       ("classifier", model)])

When I apply this pipeline step it results in:

ValueError: Found input variables with inconsistent numbers of samples: [1665, 1666]

I think a reasonable solution would be to standardize the input variable lengths after the fracdiff step to be = min_length of any one feature in X.

My questions:

  1. Is this a reasonable assumption / approach?
  2. Can you advise on how I would go about doing this in a sklearn pipeline step/transformation?
scikit-learn machine-learning python
1 Answers

I found the issue. I was using the wrong method from the fracdiff library.

self.fracdiff_transformer = FracdiffStat()

self.clf = Pipeline(
                steps=[("preprocessor", self.preprocessor),
                       ("simple_imputer", simple_imputer),
                       ("fracdiff", self.fracdiff_transformer),
                       ("classifier", model)])

This resolved the issue.

user_175821530
Related