Design for failure for a SQS-lambda that depends on a external api service

Viewed 23

I have a lambda function that asks for an external resource and we've been told the external service api could be down sometimes due maintenance, hardware issues, natural disasters, etc. And we really need the data provided by that service (we cannot obtain that data from other service). Something that comes to my mind is that we need to really wait until the service is up again.

The following handler is invoked by a normal queue.

export const handler = async (event: SQSEvent): Promise<SQSBatchResponse> => {
    const failedMessageIds: string[] = [];
    try {
        // This could fail!
        const { data } = await axios.get(`some-external-api-url/${resourceId}`);

        // Send SQS message to another queue with given data...
    } catch(err) {
        failedMessageIds.push(record.messageId);
    }
    return { batchItemFailures: failedMessageIds.map((id) => ({ itemIdentifier: id })) };
}

I've been thinking in adding a Dead Letter Queue (DLQ) to handle failed messages. So that DQL could store missing messages, wait up to 15 minutes and finally make the messages available to the consumers and lambda can pull them out and try again...

Perhaps the source queue can call this lambda and the DLQ could also call it to reuse the same lambda in both cases... Or you prefer having two different lambdas? and why?

Is this a good approach? or what would you do?

0 Answers
Related