I have simple lambda in NodeJS that always fails that consumes messages from SQS (code below).
As far as I understand, when the lambda fails, the SQS still waits visibility timeout before it retries to redeliver the message again. Is there a way to stop message visibility timeout when the processing lambda failed?. I want message, that failed to be processed by lambda, to be redelivered immediately. Note that I am only interested in SQS-Lambda integration, I don't care about different ways to process messages.
My case: in my project the lambda sometimes won't even start, because it is missing some assets (for example because of dependency upgrade). Therefore, I can't manually call ChangeMessageVisibility manually. The SQS queue has visibility timeout of 20 minutes, and there are 5 deliveries attempts before the message is moved into DLQ. I have alarm that notifies me when DLQ receives any message, so I know there is something wrong with the lambda.
Because of the visibility timeout, it takes almost 2 hours!! before I receive the notification.
Example CDK project that has 5 redeliveries and visibility timeout of 5 minutes (so it takes almost half an hour to move the message into DLQ) is here:
const env = {
account: process.env.CDK_DEFAULT_ACCOUNT,
region: process.env.CDK_DEFAULT_REGION,
};
const app = new App();
const stack = new Stack(app, 'sqs-lambda', { env });
const dlq = new Queue(stack, 'dlq', {});
const q = new Queue(stack, 'queue', {
deadLetterQueue: {
queue: dlq,
maxReceiveCount: 5,
},
visibilityTimeout: Duration.minutes(5),
});
const fn = new Function(stack, 'lambda', {
code: Code.fromInline("exports.handler = () => { console.log('executed'); throw new Error('error'); }"),
runtime: Runtime.NODEJS_16_X,
handler: 'index.handler',
events: [new SqsEventSource(q, {
batchSize: 1,
})],
})