JMS Consumer blocking other JMSXGroups

user_1546284 0
Viewed 467

I'm trying to figure out if there is anyway i can impact the Consumer's message processing order when a message is rolled back on to the queue. I have some simple code below that helps me reproduce the issue. I am simply pushing messages onto a queue with different JMSXGroupIds in a certain order:

  1. "A1" (JMSXGroupId: 1)
  2. "B1" (JMSXGroupId: 2)
  3. "A2" (JMSXGroupId: 1)
  4. "C1" (JMSXGroupId: 3)
  5. "B2" (JMSXGroupId: 2)

The code makes A1 rollback (it initially retries the message 3 times) and it goes back on to the queue with a delay. However, the consumer then waits until it can pick up A1 again (after waiting the delayed amount of time), meaning B1 and C1 groups are blocked behind A1 and never get processed.

Ideally what i was hoping for is that when A1 is put back on the queue and told to wait, the Consumer would pick up B1 and C1... what i'm ultimately trying to do is stop one JMSXGroup blocking the others on a Consumer. Also, its probably worth adding that i need to keep message sequence order for A (A1,A2,A3...) and was hoping to do that by leaving them on the queue, rather than having to build some management solution for exceptions.

onException(Exception.class)
            .log("Exception Caught !! ")
            .redeliveryDelay("1000")
            .maximumRedeliveries(3)
            .handled(false)
            .markRollbackOnly()
            .log("log:output");

    from("amq:queue:mailbox?concurrentConsumers=1")
            .to(logEndpoint)
            .process(exchange -> {
                if(exchange.getIn().getBody(String.class).contains("A")) {
                    throw new Exception("Found A");
                }
            });

I am using a Java based Apache Camel micro-service with transacted routes. Nothing really out of the ordinary but i can supply more detail/configuration details if needed.

Thanks in advance

spring-boot apache-camel jms java
3 Answers

Strict ordering on a queue is bound to run into issues like this since the queue must adhere to its fundamental first-in-first-out (i.e. FIFO) semantics. Even if B1 was picked up immediately after A1 failed you'd have to wait to consume A2 until A1 was consumed in order to preserve order, and since the queue must be consumed FIFO that would block the consumption of C1 and any of the other messages behind it.

user_83819460

You are using Camel's redelivery which will only redelivery calling the processor method (eg where it failed) and not the entire route.

You may want to look at using JMS transactions and let the message rollback all the way to the JMS broker, and configure redelivery settings on the message broker.

If you have a copy of the Camel in Action book then I would suggest that you read the transaction chapter, and the error handling chapter.

user_4064290

Your problem is because of the concurrentConsumers=1 on the consumer endpoint. If you have just one consumer, it is impossible to process the JMS groups in parallel.

With this limitation you effectively have an Exclusive Consumer and the JMS group header has no effect because there is only one consumer to process messages anyway.

The JMSXGroupId header makes sure that all messages with the same group ID are processed by the same consumer. So if you would have 3 consumers, the 3 groups in your example could be processed in parallel even if message A1 "blocks" the consumer of group A for some time.

However, when you have less consumers than groups, it is of course the case that one message can block other groups, simply because one consumer handles multiple groups.

user_80355820
Related