Fastest way to transfer buffer/texture data from GPU to CPU

As the question implies, I'm trying to transfer some buffer data from the gpu to the cpu and I want to do it fast.

Specifically, I'd like to transfer a 640x480 float buffer in less than 1ms.

Question 1: Is this possible in less than 1ms?

Whether it is or not, I'd like to find out what the fastest way is. Everything I've tried up to this point is by using FBOs. Here are the different methods and their respective average time for transfer. All of these are run right after binding to the FBO and rendering on the textures. As I'm no expert, there might be mistakes in my code or I might be doing unnecessary steps so please let me know. The transfers, however, have all been checked to be successful. I transfer everything to cv::Mat objects.

1)Using glReadPixels - < 3.1ms

   glBindTexture(GL_TEXTURE_2D,depthTexture);
   glReadPixels(0, 0, width, height, GL_DEPTH_COMPONENT, GL_FLOAT,mat.data);

2)Using glGetTexImage - <2.9ms

glBindTexture(GL_TEXTURE_2D,depthTexture);
glGetTexImage(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, GL_FLOAT, mat.data);

3)Using PBO with glGetTexImage - <2.3ms

glBindBuffer(GL_PIXEL_PACK_BUFFER, pbo);
glBindTexture(GL_TEXTURE_2D, depthTexture);
glGetTexImage(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT, GL_FLOAT, 0);
mat.data = (uchar*)glMapBuffer(GL_PIXEL_PACK_BUFFER, GL_READ_ONLY); 

Before I go on, I understand that I don't use PBOs to their full potential since I immediately call glMapBuffer but there is no other process for the cpu to do at the moment. The texture is drawn the moment I have the necessary data and the texture data is necessary for me to move on. Despite all this, PBOs still seem faster.

Here's something interesting(to me at least). These are measured in debug mode. In release mode they are all 1ms slower.

Question 2: Why are they slower in release mode? Can I change this?

Question 3: Are there any other ways I can try to do the transfer?

Extra notes on Q3:

1. I read somewhere that the integrated graphics card can have faster access. Is this a thing? How would I make use of this? Is this connected to GL_INTEL_map_texture?

2. I barely know what CUDA is but it seems there is a way to do the transfer faster using it. Is this true?

3. Will reading the depth buffer instead of a texture be faster?