Automatize adjustment of the Shadow Map Projection (by projection the points of interest onto the near clipping plane of the light source)

I am working with OpenGL so Coding in C++.

Task: I have a simple scene where I have one object, a one-point source of light, and a camera. Now I am moving the light around the object and up. When I reached all positions I want, I rotate the object and/or move the camera upwards. When I got all the positions of the light and the camera, I switch the object. Of course, all the time the lighting and the shadows should be correct. But currently, this is not the case. Which leads me to the...

Problem: Sometimes there are shadows in the background where actually shouldn't be one image with the wrong shadow in the back

I expect that the shadow map is not big enough to capture this part of the scene. Because of multiple changes in the scene, I want to automate the definition of the size of the shadow map.

What I have so far: I define the maximal bounding box around every object to know the maximal dimensions here. Through drawing the bounding box and debugging I can say that this is calculated correctly. (I know that I have to take the extent of the base surface into account as well but to clarify if my algorithm works well, I leaf that out for a second)

To geht the extents on the near clipping plane, I project them onto the near clipping plane of the light source to get dimantions for the shadow map I need to use.For the Projection I span the base for the plane with the up and the right vector of the plane. The projection matrix I calculate with A * (A^T * A)^(-1) * A^T When I now switch the view so that I see the scene from the perspective of the light, I see what I expect: The whole picture is just the object.

object from the light perspective

So far so good. But when I move the light the to right or the left, parts of the image move out of the field of view.

light moved to the right

When I move the light up the object gets swaged.

light moved up

Here is my code to adjust the lightsource_projection:

void CGRenderer::adjustLight()
    {
        glm::mat2x3 A(9.0f);
        A[1] = lightsource_camera.getUp();
        A[0] = lightsource_camera.getRight();

        //projection matrix = A(ATA)^-1 AT.
        glm::mat3 P = A * glm::inverse(glm::transpose(A) * A) * glm::transpose(A);
std::vector<glm::vec3> bbPoints;
        bbPoints.push_back(glm::vec3(object.boundingBoxViewSpace[0][0], object.boundingBoxViewSpace[0][1], object.boundingBoxViewSpace[0][2]));
        bbPoints.push_back(glm::vec3(object.boundingBoxViewSpace[1][0], object.boundingBoxViewSpace[0][1], object.boundingBoxViewSpace[0][2]));
        bbPoints.push_back(glm::vec3(object.boundingBoxViewSpace[0][0], object.boundingBoxViewSpace[0][1], object.boundingBoxViewSpace[1][2]));
        bbPoints.push_back(glm::vec3(object.boundingBoxViewSpace[1][0], object.boundingBoxViewSpace[0][1], object.boundingBoxViewSpace[1][2]));
        bbPoints.push_back(glm::vec3(object.boundingBoxViewSpace[0][0], object.boundingBoxViewSpace[1][1], object.boundingBoxViewSpace[0][2]));
        bbPoints.push_back(glm::vec3(object.boundingBoxViewSpace[1][0], object.boundingBoxViewSpace[1][1], object.boundingBoxViewSpace[0][2]));
        bbPoints.push_back(glm::vec3(object.boundingBoxViewSpace[0][0], object.boundingBoxViewSpace[1][1], object.boundingBoxViewSpace[1][2]));
        bbPoints.push_back(glm::vec3(object.boundingBoxViewSpace[1][0], object.boundingBoxViewSpace[1][1], object.boundingBoxViewSpace[1][2]));

        glm::vec3 tmp;
        float x_min = 0.f;
        float x_max = 0.f;
        float y_min = 0.f;
        float y_max = 0.f;
        for (glm::vec3 v : bbPoints)
        {
            tmp = v * P;

            if (tmp.x < x_min)
                x_min = tmp.x;
            if (tmp.x > x_max)
                x_max = tmp.x;
            if (tmp.y < y_min)
                y_min = tmp.y;
            if (tmp.y > y_max)
                y_max = tmp.y;
        }

        parameter.lightprojection_x_min = x_min;
        parameter.lightprojection_x_max = x_max;
        parameter.lightprojection_y_min = y_min;
        parameter.lightprojection_y_max = y_max;

        lightsource_projection = glm::ortho(parameter.lightprojection_x_min, parameter.lightprojection_x_max,
            parameter.lightprojection_y_min, parameter.lightprojection_y_max,
            parameter.lightprojection_z_min, parameter.lightprojection_z_max);
}

Here my shadow pass function where I adjust the light and use the newly calculated lightsource_projection:

    void CGRenderer::shadowmap_pass()
{
    glBindFramebuffer(GL_FRAMEBUFFER, framebuffers.shadowmap_buffer);
    
    glViewport(0, 0, shadowmap_width, shadowmap_height);

    glClear(GL_DEPTH_BUFFER_BIT | GL_COLOR_BUFFER_BIT);

    model = glm::scale(glm::mat4_cast(parameter.globalRotation), glm::vec3(parameter.modelScalation));
    model = glm::rotate(glm::mat4(1.f), glm::radians(parameter.modelRotation), glm::vec3(0.f, 1.f, 0.f)) * model;

    glm::mat4 shadow_view = lightsource_camera.getViewMatrix();

    glm::mat4 shadow_model_view = shadow_view * model;


    adjustLight();

    shader->use();
    glUniformMatrix4fv(locs.modelViewProjection, 1, GL_FALSE, glm::value_ptr(lightsource_projection * shadow_model_view));

    glUniformSubroutinesuiv(GL_VERTEX_SHADER, 1, &locs.placementVS);
    glUniformSubroutinesuiv(GL_FRAGMENT_SHADER, 1, &locs.depthmapFS);

    glBindVertexArray(basesurface.vao);
    glDrawArrays(GL_TRIANGLES, 0, basesurface.vertsToDraw);

    glBindVertexArray(object.vao);
    glDrawArrays(GL_TRIANGLES, 0, object.vertsToDraw);
}

Questions: Why is it not updating the projection correctly? How can I fix that problem? Do I need to change the view matrix to solve the problem?