XGBoost gives nondeterministic predictions in version <1.0. for INT input

Viewed 104

I've trained a classification model in 0.9:

param = {
    'objective': 'multi:softprob',  
    'num_class': 9,
    'booster': 'dart',
    'eta': 0.3,
    'gamma': 0,
    'max_depth': 6,
    'alpha': 0,
    'lambda': 1,
    'colsample_bylevel':0.8,
    'colsample_bynode': 0.8,
    'colsample_bytree': 0.8,
    'normalize_type': 'tree',
    'rate_drop': 1.0,
    'min_child_weight': 5,
    'subsample': 0.5,
    'num_parallel_tree': 1,
    'tree_method': 'approx'
 }
model = xgb.train(param, D_train, num_boost_round=1)
model.save_model('./model.bst')

Here is the sample training data: {f1:1, f2:1, label:"1"} Both f1 and f2 are integers.

During prediction, the results are nondeterministic with the same input. Sometimes (~1 out of 10 times) it gives equal probability for every output class.

This issue is gone when switching to XGB 1.0: Use XGB 1.0 to make predictions on a model trained in 0.9.

imported_model = xgb.Booster(model_file='./model.bst')
encoded=[[[0, 0]]
prediction_input = xgb.DMatrix(
       np.array(encoded).reshape(2, -1), missing=None)

for i in range(1000):
  outputs = imported_model.predict(prediction_input)
  print(outputs)

Does anyone know the root cause?

0 Answers
Related