AWS Cognito SignIn latency too high

I am using aws cognito for sign-up and sign-in in my application. My application backend is hosted on aws lambda and is fronted my api-gateway backed api's. For example am using "amazon-cognito-identity-js" library and calling sign up from my nodejs lambda in the backend. The api i have exposed is /user/login to the users.

The code in my backend lambda calling sign-in is as follows:

exports.signIn = (email, password) =>
    new Promise((resolve, reject) => {
        const authenticationDetails = new Cognito.AuthenticationDetails({
            Username: email,
            Password: password
        });

        const cognitoUser = new Cognito.CognitoUser({
            Username: email,
            Pool: userPool
        });

        cognitoUser.authenticateUser(authenticationDetails, {
            onSuccess: result => {
                resolve({
                    data: {
                        token: result.getIdToken().getJwtToken(),
                        refreshToken: result.getRefreshToken().getToken()
                    }
                })
            },
            onFailure: err => {
                console.log(err);
                reject({
                    data: {
                        errorCode: err.code,
                        errorMessage: err.message
                    },
                    statusCode: 400
                })
            }
        });
});

The API is working fine as expected and I am able to get the token back.

The issue I am facing is the high latency of this API. The latency is of the order of 5-8 seconds which is too high. I have tested and this high latency cannot be attributed to lambda cold start.

Can someone help me to understand why am I getting such high latency?