multiple GPU encoding use FFMPEG under multiple threads environment

Here is the application situation. I want use 4 NV GPUs to carry out some calculation work (CUDA based), and encoding work (FFMPEG &NVEC), it works fine when they are in serial. But when I want them to be parallel. like this

void encodingwork(){
            #pragma omp parallel for num_threads(4)
                for (int i = 0; i < 4; i++) {
                    cudaSetDevice(i);
                    encodingwork();
                }
        }

        int main(){
            thread thd;
            for (int k = 0; k < N; k++) {
#pragma omp parallel for num_threads(4)
                for (int i = 0; i < 4; i++) {
                    cudaSetDevice(i);
                    cudawork();
                }
                if (k != 0)
                    thd.join();
                thd = thread(encodingwork);
            }
            thd.join();
        }

an error will be thrown out from ffmpeg. just like

[AVHWDeviceContext @ 000002334959ac80] cu->cuMemFree((CUdeviceptr)data) failed -> CUDA_ERROR_ILLEGAL_ADDRESS: an illegal memory access was encountered

[hevc_nvenc @ 00000230c956b200] EncodePicture failed!: generic error (20):
Error sending a frame for encoding: -1313558101 (Unknown error occurred)

can anybody tell me what's wrong with this?