Python boto3 and SQS not receiving messages

Question

I am trying to implement sending and receiving messages using Amazon's SQS in Python nothing i do makes the receiving end of my code retrieve my sent messages, what am i doing wrong?

send.py:

SESSIONS_ACCESS_KEY = "************" 
SESSIONS_SECRET_KEY = "************"
sess = boto3.session.Session(aws_access_key_id=SESSIONS_ACCESS_KEY, aws_secret_access_key=SESSIONS_SECRET_KEY, region_name='eu-central-1')
sqs = sess.resource("sqs")
queue = sqs.get_queue_by_name(QueueName='myTestQueue.fifo')
response = queue.send_message(
    MessageBody="TEST 123",
    MessageGroupId='messageGroup1'
)

print(response.get('MessageId'))
print(response.get('MD5OfMessageBody'))

rec.py:

import time
import boto3
   
SESSIONS_ACCESS_KEY = "************" 
SESSIONS_SECRET_KEY = "************"
sess = boto3.session.Session(aws_access_key_id=SESSIONS_ACCESS_KEY, aws_secret_access_key=SESSIONS_SECRET_KEY, region_name='eu-central-1')
sqs = sess.resource("sqs")

queue = sqs.get_queue_by_name(QueueName='myTestQueue.fifo')

# Process messages by printing out body
while True:
    messages = queue.receive_messages()
    print('Amount of existing Queue messages',len(messages))
    for message in messages:
        print('msg:',message.body)
        message.delete()
    time.sleep(5)

When running the send.py python file i get the following:

1451fdb5-0f95-45d6-b1c1-76d9092fb49a
911e12e2292eb0914f39540ae513721c

But the rec.py python file i keeps getting a 0 messages in queue notification:

Amount of existing Queue messages 0
Amount of existing Queue messages 0
Amount of existing Queue messages 0
Amount of existing Queue messages 0

What am i doing wrong? should i set the MessageGroupId in the receiving end as well? should i use other commands for either sending or receiving the message?

user_174777 · Answer

When I ran your send.py code, I received an error message:

botocore.exceptions.ClientError: An error occurred (InvalidParameterValue) when calling the SendMessage operation: The queue should either have ContentBasedDeduplication enabled or MessageDeduplicationId provided explicitly

This is a feature of FIFO queues that avoids duplicate messages. I'm not sure what you had configured on your queue, so I activated ContentBasedDeduplication and then received a log message similar to what you showed.

I was then able to reproduce your situation where the messages were not received.

However, given that ContentBasedDeduplication was enabled, I then modified send.py:

response = queue.send_message(
    MessageBody="TEST 123"+ str(random.random()),
    MessageGroupId='messageGroup1'
)

When sending a message with this code, the message was successfully received by rec.py.

Bottom line: The FIFO queue was filtering duplicate messages, which is probably how you configured it.

Python boto3 and SQS not receiving messages

1 Answers