I am running Azure Databricks AutoML using 12.2 cluster that claims handling the imbalanced dataset. Although, it is not in my input, this cluster adds extra columns as _automl_sample_weight_ac91, _automl_split_col_636d. It drops_automl_split_col_636d column before training, but keeps the _automl_sample_weight_ac91 column. After registering the model, I am trying to predict very separate dataset than the one I used on training... Apparently, registered automl model adds _automl_sample_weight_ac91 as column, but my very separate test data frame has not sample weight column, thus gives the following error during prediction:
PythonException: 'mlflow.exceptions. MlflowException: Model is missing inputs ['_automl_sample_weight_ac91'].'. Full traceback below:
How can I solve this error related to _automl_sample_weight_ac91)?
Thanks