AWSLambdaClient invoke giving exception: com.amazonaws.SdkClientException: Unable to execute HTTP request: The target server failed to respond

Viewed 4082

I have an existing aws lambda function and I am trying to invoke it using AWSLambdaClient (library: aws-java-sdk-lambda). I get the response if the runtime for lambda is less than 5 mins. But if the lambda runtime is more than 5 mins I get the exception:

com.amazonaws.SdkClientException: Unable to execute HTTP request: The target server failed to respond

What configuration I am missing while setting up lambda client ?

Full stack trace:

Exception in thread "main" com.amazonaws.SdkClientException: Unable to execute HTTP request: The target server failed to respond
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1163)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1109)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:758)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:732)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:714)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:674)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:656)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:520)
at com.amazonaws.services.lambda.AWSLambdaClient.doInvoke(AWSLambdaClient.java:3154)
at com.amazonaws.services.lambda.AWSLambdaClient.invoke(AWSLambdaClient.java:3121)
at com.amazonaws.services.lambda.AWSLambdaClient.invoke(AWSLambdaClient.java:3110)
at com.amazonaws.services.lambda.AWSLambdaClient.executeInvoke(AWSLambdaClient.java:1735)
at com.amazonaws.services.lambda.AWSLambdaClient.invoke(AWSLambdaClient.java:1707)
at LambdaTest.main(LambdaTest.java:48)
Caused by: org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at com.amazonaws.http.protocol.SdkHttpRequestExecutor.doReceiveResponse(SdkHttpRequestExecutor.java:82)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1285)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1101)
... 12 more

I have already increased my lambda max timeout to 15 mins on aws lambda console. Also I have made code changes and set ClientExecutionTimeout, ConnectionTimeout, RequestTimeout and SocketTimeout to 15 mins (900 sec) in AWSLambdaClient configuration.

ClientConfiguration configuration = new ClientConfiguration()
            .withMaxErrorRetry(0)
            // These is measured in milliseconds.
            .withClientExecutionTimeout(900 * 1000)
            .withConnectionTimeout(900 * 1000)
            .withRequestTimeout(900 * 1000)
            .withSocketTimeout(900 * 1000);
AWSLambda lambdaClient = AWSLambdaClientBuilder
            .standard()
            .withRegion(region)
            .withClientConfiguration(configuration)
            .withCredentials(new AWSStaticCredentialsProvider(creds)).build();
InvokeRequest request = new InvokeRequest()
            .withFunctionName(functionName)
            .withInvocationType(InvocationType.RequestResponse)
            .withLogType(LogType.Tail);
InvokeResult result = lambdaClient.invoke(request);
String responseLog = new String(Base64.getDecoder().decode(result.getLogResult()), StandardCharsets.UTF_8);
System.out.println(responseLog);

I am using apache HttpClient: 4.5.5

I saw some question https://stackoverflow.com/a/44703601/9669982 I tried some configurations also:

configuration.setValidateAfterInactivityMillis(900 * 1000);

Also to try more I also used:

configuration.setConnectionMaxIdleMillis(900*1000);
configuration.setUseTcpKeepAlive(true);

But no luck.

On AWS Cloudwatch I see that my lambda function is running fine and full task was executed and no trace of any kind of error was there on cloudwatch but I am getting the above exception.

I can't get what I am missing.

1 Answers

As you already know issue

I get the response if the runtime for lambda is less than 5 mins. But if the lambda runtime is more than 5 mins I get the exception:

Your connections that are kept alive by the connection manager become stale.That is, the target server shuts down the connection on its end without HttpClient being able to react to that event.

You need to add setHttpRequestRetryHandler for your HTTP client as answered in following link.

Related