I am running some tests on my Ubuntu workstation. These benchmarks start with populating a queue, which runs very slowly:
import pika
import datetime
if __name__ == '__main__':
try:
connection = pika.BlockingConnection(pika.ConnectionParameters(
host='localhost'))
channel = connection.channel()
channel.queue_declare(queue='hello_durable', durable=True)
started_at = datetime.datetime.now()
properties = pika.BasicProperties(delivery_mode=2)
for i in range(0, 100000):
channel.basic_publish(exchange='',
routing_key='hello',
body='Hello World!',
properties=properties)
if i%10000 == 0:
duration = datetime.datetime.now() - started_at
print(i, duration.total_seconds())
print(" [x] Sent 'Hello World!'")
connection.close()
now = datetime.datetime.now()
duration = now - started_at
print(duration.total_seconds())
except Exception as e:
print(e)
It takes more than 30 seconds to send 10K messages. The workstation has 12 cores, which are not busy, according to top command. There are over 8Gb of free memory. It does not matter much whether the queue is durable.
How can we speed up sending messages?