Let's say I am coming from a left-handed world space. My positive Z is into the screen, and my Y is up, the same as OpenGL's default in clip space.
It is my understanding that, when coming from a right-handed world space (which is somewhat required when using older functions such as gluLookAt which maps the forward vector to the -Z axis) the Z co-ordinated is flipped before computing the projection matrix (orthographic/perspective). This is due to the fact that a right-handed system would have negative Z into the screen, which needs to be flipped before being passed to OpenGL.
I have confirmed that, for a function such as orthoLH_ZO, no flipping takes place. In fact, the matrix is a "standard" orthographic projection that also works in e.g. Vulkan i.e. it keeps the handedness (except for the fact that Y would be down in Vulkan so bottom and top would switch).
I would therefore expect glm::frustumLH_ZO to also be such a "standard" matrix since no flip needs to take place. However, when looking at its source code, elements [2][0] and [2][1] are negated compared to e.g. Vulkan's frustum perspective projection matrix (discussed here at 11:49).
These elements are:
Result[2][0] = (right + left) / (right - left);
Result[2][1] = (top + bottom) / (top - bottom);
instead of my expected:
Result[2][0] = -(right + left) / (right - left);
Result[2][1] = -(top + bottom) / (top - bottom);
I don't believe this is due to a Z-flip, as that should also cause e.g. element [2][3] to be -1 and not +1.
Why are these entries in the matrix flipped?