I have added embedded Python scripting ability to my C++ app. Users write a script, and then my passes values into that script, executes it, and receives the results. I use Py_CompileString() to compile their script PyImport_ExecCodeModule() to import the compiled script, and PyObject_GetAttrString() to find a specially named function ('python_main') and execute that function and get the result. This all works fine.
Recompiling the code when the user changes the script also works, as the changes to the Python source are reflected in the Python output. What I find disturbing is that I am not able to fully discard the code modules performing the Python recompile. (Maybe this is because I'm an experienced C++ programmer but a newbie when it comes to Python.)
The simplified sample code below illustrates the problem. While the ref count for the compiled script is 1, the ref count for the module, the function attribute, and the result value are all 2. I would expect these to be 1 so that, when I can Py_DECREF() on these objects and fully release them, ensuring they are truly destroyed and that everything is "starting from scratch" after I recompile the Python code.
Py_Initialize();
// compile and execute python_main() in the following script
/*
def python_main():
print('python_main called')
return -100
def main():
print('**SCRIPT** main called')
def __init_(self):
print('**SCRIPT** init called')
def __del__(self):
print('**SCRIPT** del called')
if __name__ == '__test__':
print('**SCRIPT** __name__ == __test__')
main()
*/
PyObject* codeObj = NULL;
PyObject* moduleObj = NULL;
PyObject* funcObj = NULL;
PyObject* valueObj = NULL;
// compile the script (new reference)
printf("Compile Script...\n");
codeObj = Py_CompileString(sPythonCode, "myfile.py", Py_file_input);
if (codeObj != NULL) {
printf("Code Object Ref Count = %zd\n", Py_REFCNT(codeObj));
printf("Import Module...\n");
moduleObj = PyImport_ExecCodeModule("__test__", codeObj);
if (moduleObj != NULL) {
printf("Module Object Ref Count = %zd\n", Py_REFCNT(moduleObj));
printf("Get Function Object...\n");
funcObj = PyObject_GetAttrString(moduleObj, "python_main");
if (funcObj) {
printf("Function Object Ref Count = %zd\n", Py_REFCNT(funcObj));
printf("Call function object...\n");
valueObj = PyObject_CallObject(funcObj, NULL);
if (valueObj) {
printf("Function Return Value Ref Count = %zd\n", Py_REFCNT(valueObj));
}
}
}
}
if (valueObj != NULL) {
Py_DecRef(valueObj);
}
if (funcObj != NULL) {
Py_DecRef(funcObj);
}
if (moduleObj != NULL) {
Py_DecRef(moduleObj);
}
if (codeObj != NULL) {
Py_DecRef(codeObj);
}
PyGC_Collect();
printf("\n");
printf("Garbage Collect...\n");
printf("After GC: Code Object Ref Count = %zd\n", Py_REFCNT(codeObj));
printf("After GC: Module Object Ref Count = %zd\n", Py_REFCNT(moduleObj));
printf("After GC: Function Object Ref Count = %zd\n", Py_REFCNT(funcObj));
printf("After GC: Function Return Value Ref Count = %zd\n", Py_REFCNT(valueObj));
The output produced by this code is as follows. It would seem to indicate that the Code Object was destroyed, but that the module, function reference, and return value are still "lingering".
Compile Script...
Code Object Ref Count = 1
Import Module...
**SCRIPT** __name__ == __test__
**SCRIPT** main called
Module Object Ref Count = 2
Get Function Object...
Function Object Ref Count = 2
Call function object...
**SCRIPT** python_main called
Function Return Value Ref Count = 2
Garbage Collect...
After GC: Code Object Ref Count = 4388382480
After GC: Module Object Ref Count = 1
After GC: Function Object Ref Count = 1
After GC: Function Return Value Ref Count = 1
Can someone tell me if this is to be expected, or if I am doing something wrong?
Thanks!