Easy test code
glm::mat4 m = glm::rotate(glm::mat4(1.0f), 0.78f, glm::vec3(0,1,0));
while (true) {
glm::vec3 axis = glm::normalize(glm::vec3(m[0])); // right vector
PRINT_VEC("{0:.3f} {1:.3f} {2:.3f}", axis.x, axis.y, axis.z);
m = glm::rotate(m, glm::radians(5.f), axis); // 5 degrees each iteration
}
So, assume that I have a model matrix rotated from identity by 0.78 radians around the y axis, then every frame I'm going to rotate around the local right vector, which is the first column of the matrix (assume a right-handed system). Since the right vector is the axis around which I rotate, I expect it to be constant, but that's not true. I don't understand why glm::rotate also changes the rotation axis.
The output changes quite drastically so I don't think it's floating point precision errors.
0.657 -0.424 -0.623
0.643 -0.482 -0.595
0.626 -0.539 -0.563
0.608 -0.593 -0.527
0.588 -0.646 -0.487
0.566 -0.696 -0.442
0.543 -0.742 -0.393
0.518 -0.785 -0.339
0.491 -0.824 -0.281
0.464 -0.858 -0.219
0.435 -0.887 -0.153
0.406 -0.910 -0.084
0.377 -0.926 -0.012
0.347 -0.936 0.063
0.319 -0.937 0.140
0.292 -0.931 0.218
0.267 -0.917 0.296
0.244 -0.895 0.374
0.224 -0.864 0.450
0.208 -0.826 0.524

