How Peek works in a Partition enabled Service Bus Queue?

user_10403902 0
Viewed 498

I understand from the Microsoft docs that during the first Peek() operation, any one of the available message brokers respond and send their oldest message. Then on subsequent Peek() operation, we can traverse across the partitions to peek every message with increased sequence number.

My question is, during the very first Peek() operation, I will get a message from any of the first responded partitions. Is there a guarantee that I can peek all the messages from the queue?

In a much simpler way, there are three Partitions: Partition "A" has 10 messages with sequence number from 1 to 10. Partition "B" has 10 messages with sequence number from 11 to 20. Partition "C" has 10 messages with sequence number from 21 to 30.

Now if i perform Peek() operation, if Partition "B" responds first, the first message that I'll get is a message with sequence number 11. Next peek operation will look for a message with incremented sequence number. Won't I miss out messages from Partition "A" which has sequence numbers 1-10 which peek operation can never reach since it always searches for the incremented sequence number?

UPDATE

QueueClient queueClient = messagingFactory.CreateQueueClient("QueueName", ReceiveMode.PeekLock);
BrokeredMessage message = null;
while (iteration < messageCount)
{
  message = queueClient.Peek(); // According to docs, Peeks the oldest message from any responding broker, and next iterations peek the message with incremented sequence number
  if (message == null)
    break;
  Console.WriteLine(message.SequenceNumber);
  iteration++;
}

Is there a guarantee that I can browse all the messages of a partitioned queue using the snippet above?

azure-servicebus-queues azureservicebus azure c#
2 Answers

There is no guarantee that the returned message is the oldest one across all partitions.

It therefore depends which message broker responds first, and the oldest message from that partition will be shown. There is no general rule as to which partition will respond first in your example, but it is guaranteed that the oldest message from that partition is displayed first.

If you want to retrieve the messages by sequence number, use the overloaded Peek method Peek(Sequencenumber), see: https://docs.microsoft.com/en-us/azure/service-bus-messaging/message-browsing

user_143448210

For partitioned entities, the sequence number is issued relative to the partition.

[...]

The SequenceNumber value is a unique 64-bit integer assigned to a message as it is accepted and stored by the broker and functions as its internal identifier. For partitioned entities, the topmost 16 bits reflect the partition identifier. (https://docs.microsoft.com/en-us/azure/service-bus-messaging/message-sequencing)

So you cannot compare sequence numbers across partitions to see which one is older.

As an example, I just created a partitioned queue and put a couple of messages into two partitions (in Order):

1. Partition 1, SequenceNumber 61924494876344321
2. Partition 2, SequenceNumber 28991922601197569
3. Partition 1, SequenceNumber 61924494876344322
4. Partition 1, SequenceNumber 61924494876344323
5. Partition 2, SequenceNumber 28991922601197570

Browse/Peek messages: Available only in the older WindowsAzure.ServiceBus library. PeekBatch does not always return the number of messages specified in the MessageCount property. There are two common reasons for this behavior. One reason is that the aggregated size of the collection of messages exceeds the maximum size of 256 KB. Another reason is that if the queue or topic has the EnablePartitioning property set to true, a partition may not have enough messages to complete the requested number of messages. In general, if an application wants to receive a specific number of messages, it should call PeekBatch repeatedly until it gets that number of messages, or there are no more messages to peek. (https://docs.microsoft.com/en-us/azure/service-bus-messaging/service-bus-partitioning, Emphasis added)

As such, you should be able to repeatedly call Peek / PeekBatch to eventually get all the messages. At least, if you use the official SDKs.

user_3363780
Related