C++ 3D Scene distorts when looking up or down

I have a problem that is identical to this problem, however the solution provided doesn't solve my issue.

I'm developing this using the PixelGameEngine from javidx9, so I have implemented all these functions myself.

This is the code for the LookAt function:

static Matrix4 LookAtMatrix(const Vector3& pos, const Vector3& target) {
        if(pos == target) { return LookAtMatrix(pos, target + Vector3(.01f, 0, 0)); }

        //get new forward direction
        Vector3 newFor = (target - pos).normalized();

        //get right direction
        Vector3 newRight; 
        if(newFor == Vector3::UP || newFor == Vector3::DOWN) { 
            newRight = Vector3::RIGHT; 
        } else {
            newRight = (Vector3::UP.cross(newFor)).normalized(); 
        }

        //get up direction
        Vector3 newUp = newRight.cross(newFor); 

        //make look-at matrix
        return Matrix4(
            newRight.x, newRight.y, newRight.z, 0,
            newUp.x,    newUp.y,    newUp.z,    0,
            newFor.x,   newFor.y,   newFor.z,   0,
            pos.x,      pos.y,      pos.z,      1
        );
    }

You may notice that some of my cross products are flipped when compared to other implementations, but it works with moving the camera around just fine.

Here is a gif of the warping:

I don't understand what's going wrong, and my friend who I'm working on this with went at it for weeks and couldn't find a solution. I don't know what else could be causing the problem.

A quick run down of our rendering process, in case that's the issue is

//project points to camera space
//clip triangles against view plane
//project triangles to screenspace
//clip triangles against camera's view fulcrum
//draw triangles

If more detail is needed anywhere I'll be happy to provide it. Thanks.