Configurable joint, rotation not acting as expected

I've take the Crawler project example and rework it for a personal project. And I have a huge issue making work the joints of the lower legs (not the upper legs thought).

I had to apply local transformation like this, so that the object are all in the same direction. Then I've edit the axis in the configurable joint, only the Z axis is limited, the others are locked. Then I've applied a 90 limit angle rotation.

Thing is, in what ever constraint I use for the axis, when I apply an angle, I still get the same angle limit. I use a model to predict between -1 and 1 an output for the angle, which is mapped in a function to give 0 or 180 Degrees.

    public void SetJointTargetRotation(float x, float y, float z)
    {
        x = (x + 1f) * 0.5f;
        y = (y + 1f) * 0.5f;
        z = (z + 1f) * 0.5f;

        var xRot = Mathf.Lerp(joint.lowAngularXLimit.limit, joint.highAngularXLimit.limit, x);
        var yRot = Mathf.Lerp(-joint.angularYLimit.limit, joint.angularYLimit.limit, y);
        var zRot = Mathf.Lerp(-joint.angularZLimit.limit, joint.angularZLimit.limit, z);

        currentXNormalizedRot =
            Mathf.InverseLerp(joint.lowAngularXLimit.limit, joint.highAngularXLimit.limit, xRot);
        currentYNormalizedRot = Mathf.InverseLerp(-joint.angularYLimit.limit, joint.angularYLimit.limit, yRot);
        currentZNormalizedRot = Mathf.InverseLerp(-joint.angularZLimit.limit, joint.angularZLimit.limit, zRot);

        joint.targetRotation = Quaternion.Euler(xRot, yRot, zRot);
        currentEularJointRotation = new Vector3(xRot, yRot, zRot);
    }

But for testing I'm not using the neural net yet, I hard code inputs like this:

bpDict[leg3Lower].SetJointTargetRotation(0, 0, 1f);

But this is what I get:

In red, the movement of the leg (down to right). In bleu, the original position of the leg. In Black, the "axis" and limit that it was supposed to make.\

And even if I fool around with the axis, it will always go to the same position when I apply 1f(inside the robot, supposed to be down) -1 (flat, supposed to be up the sky).

Am I missing something?