Can I use xgboost global model properly, if I skip step_dummy(all_nominal_predictors(), one_hot = TRUE)?

user_19559137 0
Viewed 44

I wanted to try xgboost global model from: https://business-science.github.io/modeltime/articles/modeling-panel-data.html

On smaller scale it works fine( Like wmt data-7 departments,7ids), but what if I would like to run it on 200 000 time series (ids)? It means step dummy creates another 200k columns & pc can't handle it.(pc can't handle even 14k ids)

I tried to remove step_dummy, but then I end up with xgboost forecasting same values for all ids.

My question is: How can I forecast 200k time series with global xgboost model and be able to forecast proper values for each one of the 200k ids. Or is it necessary to put there step_ dummy in oder to create proper FC for all ids?

Ps:code should be the same as one in the link. Only in my dataset there are 50 monthly observations for each id.

timetk xgboost tidymodels r
1 Answers

For this model, the data must be given to xgboost in the format of a sparse matrix. That means that there should not be any non-numeric columns in the data prior to the conversion (with tidymodels does under the hood at the last minute).

The traditional method for converting a qualitative predictor into a quantitative one is to use dummy variables. There are a lot of other choices though. You can use an effect encoding, feature hashing, or others too.

user_10786010
Related