ResourceWarning: Implicitly cleaning up

Question

I have the following code, which is called test_build, and it has a test case to save a scikit-learn model along with x_train, y_train and score data, in a tuple object to a ".pkl" file.

from build import *
import os 
import pandas as pd
import sklearn 
from sklearn import *
import unittest
from sklearn.model_selection import train_test_split
import numpy as np
import tempfile

class TestMachineLearningUtils(unittest.TestCase):

    def test_save_model(self):

        X, y = np.arange(10).reshape((5,2)), range(5) 
        model =  RandomForestClassifier(n_estimators = 300,
                                        oob_score = True, 
                                        n_jobs = -1,
                                        random_state = 123)
        X_train, X_test, y_train, y_test = train_test_split(\
                X, y, test_size=0.33, random_state=42) 
        clf = model.fit(X_train, y_train)
        score = model.score(X_test, y_test)
        dir_path = os.path.dirname(os.path.realpath(__file__))
        f = tempfile.TemporaryDirectory(dir = dir_path)
        pkl_file_name = f.name + "/" + "pickle_model.pkl"
        tuple_objects = (clf, X_train, y_train,  score)
        path_model = save_model(tuple_objects, pkl_file_name)
        exists_model = os.path.exists(path_model)
        self.assertExists(exists_model, True)



if __name__ == "__main__":
    unittest.main()

This is the content of the save_model function found in the build module I imported in my test file.

def save_model(tuple_objects, model_path): 

    pickle.dump(tuple_objects, open(model_path), 'wb')

    return model_path

The problem I am running to, is that I cannot test if the file is created within a temporal directory. It is apparently created, but it is cleaned after it has been created, from the error message I receive.

C:\Users\User\AppData\Local\Continuum\miniconda3\envs\geoenv\lib	empfile.py:798: ResourceWarning: Implicitly cleaning up

Does anyone knows a solution to this problem? How could one supress the cleaning up of a temporary directory created using the tempfile module in python?

user_3437012 · Answer

It looks to me like your code does exactly what you want it to do and you just get confused by the warning. The warning is merely telling you that you should explicitly delete the temporary directory, but that the module is so kind as to do it for you.

To delete the temporary directory yourself, either use it in a context manager or call the close method on it.

user_828237 · Answer

You simply don't. If you want the directory to outlast the scope you create "not a temporary directory".

Or more likely when testing - you create directory in test setup, fill, test, teardown, so each test is independent.

ResourceWarning: Implicitly cleaning up <Temporary Directory>

2 Answers