AWS Lambda, retries on timeout, Python SDK

user_11908107 0
Viewed 1085

I was trying to invoke a Lambda function through the Python SDK in a synchronous fashion in a Jupyter notebook. An event I am sending is such that it takes more than the maximum possible timeout limit (15 min) to complete.

I noticed that the event sometimes (not always) is being re-sent to the lambda upon the timeout error. This keeps going on and on until I shutdown the lambda by setting its concurrency to 0. This never happens if I lower the timeout limit (e.g., 10 minutes), meaning, the event is never being re-sent, there is only one invocation in the log, only one error and no activity afterwards.

What is going on? How do I rationalize these observations?

aws-lambda aws-sdk amazon-web-services python
2 Answers

Try looking at step functions, by doing this you can control the retry logic of Lambda and mark it as a failure.

IF your Lambda function is taking 15 minutes, determine whether you can break it down into smaller Lambda functions and invoke each of these in turn in your Lambda function.

user_134609330

I recommend turning on DEBUG level debugging and examining CloudWatch logs when you see it get executed more than once. I've seen this sometimes, and when I do I usually see log entries that come from the SDK code itself that tell me it has some built-in retry logic that is executing. If the call to invoke the lambda doesn't get a proper response, it may retry the call again--but it is possible the service received the original request and executed it, yet something went wrong with the response and so the caller re-at

Check out what is said at this link: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-retry-timeout-sdk/

Note: API calls can take longer than expected when network connection issues occur. Network issues can also cause retries and duplicated API requests. To prepare for these occurrences, your Lambda function must always be idempotent.

If you make an API call using an AWS SDK and the call fails, the SDK automatically retries the call. How long and how many times the SDK retries is determined by settings that vary among each SDK.

That article has tips for troubleshooting or changing config settings.

Also see https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html

user_2300550
Related