Could not load library cudnn_ops_infer64_8.dll. Error code 126 Please make sure cudnn_ops_infer64_8.dll is in your library path

You should have downloaded CUDA zip file. Go to that file, extract it and in the bin folder you will see

cudnn_adv_infer64_8.dll
cudnn_adv_train64_8.dll
cudnn_cnn_infer64_8.dll
cudnn_cnn_train64_8.dll
cudnn_ops_infer64_8.dll
cudnn_ops_train64_8.dll

Copy these files into your nvidia gpu computing toolkit\cuda\bin folder.

In my case, it is because I did not install zlib, and after I install it, it works!!

Leaving an answer to respond to Diego Rueda's comment on MADM4X's post.

I ran into the same issue: copied the cuDNN files into my CUDA toolkit install and received the Error Code 126.

You need to specifically download/copy cuDNN Version 8.1.x. If you use the latest version (8.3.x) you'll receive the error Code described in the original post.

TensorFlow doesn't seem to be as sensitive to the Toolkit's version (I'm running 11.4), but I haven't explored all of the features to make sure they work.

For more context, you can find the specific CUDA/cuDNN versions listed on TensorFlow's website.

Jupyter notebook was locking the file in my case, closing Jupyter notebook server solved it for me.

I had the same problem and just went thru the Prerequisites: https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#prerequisites-windows

Check if all the cudnn[.dll, .h, .lib]* files and zlivwapi.dll are in the system path. Once all are in path, the problem is solved.