How to send messages to dead letter queue after client error?

user_7548923 0
Viewed 344

Amazon SNS does not send messages to Dead Letter Queue (DLQ) when the subscription-http-endpoint returns a client-side-error.

I have a SNS topic which is sending notifications to a https-endpoint. I also attached a Dead Letter Queue to it and enabled delivery status logging.

When notifications can't reach the endpoint because of network timeout or a server-side-error (e.g. 500) from the endpoint, the notification gets into the retry-policy and after the configured amount of retries, the notification is sent to the DLQ. This works fine! This is how the logs looks like:

{
    "notification": {
        "messageMD5Sum": "1234",
        "messageId": "123-456-789",
        "topicArn": "arn:aws:sns:eu-west-1:1234:mysnstopic",
        "timestamp": "2021-06-08 15:15:21.454"
    },
    "delivery": {
        "deliveryId": "84b16cc6-09bd-5186-a6cb-35ff8274dc9d",
        "redrivePolicy": "{\n  \"deadLetterTargetArn\": \"arn:aws:sqs:eu-west-1:1234:mydlq\"\n}\n",
        "destination": "https://my-endpoint.xy/",
        "providerResponse": "Socket timeout in HttpClient",
        "dwellTimeMs": 30137,
        "attempts": 3
    },
    "status": "FAILURE"
}

But sometimes it happens, that the https-endpoint responds with a client-side-error (e.g. 404). In this case SNS logs the notification as "SUCCESS" and is finished. It does not go into retry-policy (which is OK) and it does not send the notification to the DLQ (not OK). Logs looks like this:

{
    "notification": {
        "messageMD5Sum": "1234",
        "messageId": "123-456-789",
        "topicArn": "arn:aws:sns:eu-west-1:1234:mysnstopic",
        "timestamp": "2021-06-08 15:24:46.146"
    },
    "delivery": {
        "deliveryId": "111-222-333",
        "redrivePolicy": "{\n  \"deadLetterTargetArn\": \"arn:aws:sqs:eu-west-1:1234:mydlq\"\n}\n",
        "destination": "https://my-endpoint.xy/",
        "providerResponse": "Not Found",
        "dwellTimeMs": 202,
        "attempts": 1,
        "statusCode": 404
    },
    "status": "SUCCESS"
}

In the documentation it says

In general, message delivery fails when Amazon SNS can't access a subscribed endpoint due to a client-side or server-side error. When Amazon SNS receives a client-side error, or continues to receive a server-side error for a message beyond the number of retries specified by the corresponding retry policy, Amazon SNS discards the message—unless a dead-letter queue is attached to the subscription.

According to the doc, I expect that notifications which gets a 404 will be not SUCCESS but FAILURE and are sent to the DLQ. How can I configure it to have the desired effect?

amazon-sns amazon-web-services
1 Answers

Sorry if I'm a bit later to answer this but according to the AWS Docs:

Make sure that your endpoint responds to the HTTP POST message from Amazon SNS with the appropriate status code. The connection will time out in 15 seconds. If your endpoint does not respond before the connection times out or if your endpoint returns a status code outside the range of 200-4xx SNS will consider the delivery of the message as a failed attempt.

So you need the endpoint to return a 503 Service Unavailable etc... for them to be registered as undeliverable in SNS

user_172007210
Related