When a CUDA stream is created on a specific device via:
int device = 1; // Example number
cudaSetDevice(device);
cudaStream_t cudaStream;
cudaStreamCreate(&cudaStream); // Similar with cudaStreamCreateWithFlags
Do I have to set the device again before deleting it, or can I just call?
// cudaSetDevice(device); // Is this needed ???
cudaStreamDestroy(cudaStream);
No, you don't have to. As for CUDA documentation:
"A kernel launch will fail if it is issued to a stream that is not associated to the current device
A memory copy will succeed even if it is issued to a stream that is not associated to the current device.
cudaEventRecord() will fail if the input event and input stream are associated to different devices.
cudaEventElapsedTime() will fail if the two input events are associated to different devices.
cudaEventSynchronize() and cudaEventQuery() will succeed even if the input event is associated to a device that is different from the current device.
cudaStreamWaitEvent() will succeed even if the input stream and input event are associated to different devices. cudaStreamWaitEvent() can therefore be used to synchronize multiple devices with each other."
So there is no mention of problems regarding cudaStreamDestroy(), and I tested it using two GPU devices, setting the first device, creating a stream on it, then setting the second device and creating another stream on it. After that, I ran a kernel on the second device, then I destroyed the second stream and first stream with no error, and without the need to set the first device again.
cudaStatus = cudaSetDevice(0);
cudaStream_t s0;
cudaStreamCreate(&s0);
cudaStatus = cudaSetDevice(1);
cudaStream_t s1;
cudaStreamCreate(&s1);
addKernel<<<1, size, 1, s1>>>(dev_c, dev_a, dev_b);
cudaDeviceSynchronize();
cudaStatus = cudaStreamDestroy(s1);
fprintf(stderr, "%s\n", cudaGetErrorString(cudaStatus));
cudaStatus = cudaStreamDestroy(s0);
fprintf(stderr, "%s\n", cudaGetErrorString(cudaStatus));
This is the result of cudaStatus.
