How to compare XGBoost model objects, one initialized/fitted and the other read from file, within a pytest unit test?

Viewed 724

I have a simple test of a class I've used to encapsulate an XGBoost model. In order to test this class I've trained an XGBoost model and saved it to file, and I want to use this trained model that I'll read from file to test my model training code. I'm unsure of how I can best go about comparing an XGBoost model that I'll train with known parameters/data with the one I've saved to file. For example, I have trained and saved an XGBoost model like so:

# specify parameters to use for training the XGBoost model
params = {
    'max_depth': 6,  # the maximum depth of each tree
    'eta': 0.25,  # the training step for each iteration
    'silent': 1,  # logging mode - quiet
    'objective': 'reg:tweedie',
    'booster': 'gbtree',
    'subsample': 0.7,
    'gamma': 0.3,  # regularization parameter
    'colsample_bytree': 0.2,
    'rate_drop': 0.3,
    'skip_drop': 0.2,
    'early_stopping_rounds': 10,
    'eval_metric': ['rmse', 'mae'],  # error evaluation for multiclass training
}

# split X and y into train and test sets
features_train, features_test, target_train, target_test = \
    train_test_split(features, target, test_size=test_percentage, random_state=31)

# package the dataset splits as input for XGBoost
dtrain = xgb.DMatrix(features_train, label=target_train)
dtest = xgb.DMatrix(features_test, label=target_test)
evallist = [(dtest, 'eval'), (dtrain, 'train')]

# train the XGBoost model
xgbooster = xgb.train(params, dtrain, training_iterations, evallist, verbose_eval=0)
pickle.dump(xgbooster, open("/path/to/fitted_model.dat", "wb"))

In the (pytest) unit test for my model class I want to test that I'm training the model as expected so I am reading this saved model from file for comparison against one that should match:

def test_xgboost_fit():

    features_train_df = pd.read_csv("/path/to/features_train.csv"))
    labels_train_df = pd.read_csv("/path/to/labels_train.csv"))
    fixture_xgbooster = pickle.load(open("/path/to/fitted_model.dat", "rb"))

    # train/fit the model
    xgbooster = mymodelclass.XGBoostModel()
    xgbooster.fit(features_train_df, labels_train_df)

    # compare the trained model against the expected model read from file
    assert xgbooster.model == fixture_xgbooster

Using a double equals here seems insufficient for comparison (otherwise I have something else amiss, as it's showing that two models with the same parameters and fitted with the same training data are not equal).

How should I go about making this comparison in my test? Or is there a better approach for testing this code?

0 Answers
Related