How to sync the device setting between pytorch(python) and libtorch(C++)?

There is a annoying problem when using a libtorch function in pytorch(python). The device setting in libtorch is k:CUDA, and its default meaning is CUDA:0. But when I use it in python pytorch, I want to use other CUDA device, not CUDA:0. I have changed CUDA_VISIBLE_DEVICES in python, but this can't effect the libtorch device settings. Is there any solution for this condition? Thanks~