SQS ReceiveMessageRequest not receiving all the messages in the queue (messages are under 10)

Aditya K 0
Viewed 30

I'm trying to retrieve messages from a queue. I understand that RecieveMessageRequest has a threshold of 10 messages but when I tried I was able to receive only 2 out 3 messages in the queue. I read many threads which said adding setMaxNumberOfMessages(10) and increasing WaitTimeSeconds will fix it(Before adding this I received only one message out of 3) but it wasn't helpful.

FYI: I'm using a standard queue and all the messages were definitely there in the queue at the time of receive message request so it shouldn't have been a polling issue.

My implementation:

List<Message> messages;

ReceiveMessageRequest receiveMessageRequest = new ReceiveMessageRequest().withQueueUrl(queueUrl)
                    .withWaitTimeSeconds(10)
                    .withMaxNumberOfMessages(10);
           
messages = sqsConfig.getSQSClient().receiveMessage(receiveMessageRequest).getMessages();
amazon-sqs spring-boot java amazon-web-services
1 Answers

I know it make not make a ton of sense but in multiple programming languages (certainly Java and Python) I've had to loop to get everything from a queue. In Java it's something like (using the V2 api's):

// note that "done" needs to be set somewhere else to stop the loop
while (!done) {
    ReceiveMessageRequest receiveMessageRequest = ReceiveMessageRequest.builder()
                    .queueUrl(queueUrl)
                    .waitTimeSeconds(20)
                    .build();
    List<Message> messages = sqsClient.receiveMessage(receiveMessageRequest).messages();

    for (Message nextMessage : messages) {
        // do something with the message

       DeleteMessageRequest deleteMessageRequest = DeleteMessageRequest
                        .builder()
                        .queueUrl(queueUrl)
                        .receiptHandle(nextMessage.receiptHandle())
                        .build();

       sqsClient.deleteMessage(deleteMessageRequest);
   }
}

From what I've read it is because there are ultimately many servers behind SQS and a single call doesn't hit all of them - it takes multiple calls. This is not a busy loop as the waitTimeSeconds does block if there is nothing to do.

Try something like this to read SQS. It's not quite as elegant but it does work.

stdunbar0
Related