Create Flux from messages on SQS queue

Viewed 511

I'm trying to create a Flux from incoming messages received from a queue.

For instance, if I'm using Amazon SQS how do I achieve to write the following code:

Flux<String> messages = connectionToSQS.receiveFromQueue(queueName);

messages.map(s -> log.info("message: {}", s).subscribe();

After experimentation, I found the following issues:

  • How do I keep requesting messages from the queue (loop forever)? Do I create one thread that has a loop that keeps on requesting from the queue?
  • How do I make the Flux cold? I don't want to request messages from SQS unless the consumer asks for it. This allows me to use backpressure.

First pass over this problem yielded something like the following code as per Reactor documentation:

Flux<String> bridge = Flux.create(sink -> {
    myEventProcessor.register(new MyEventListener<String>() { 
        public void onDataChunk(List<String> chunk) {
            for(String s : chunk) {
                sink.next(s); 
            }
        }

        public void processComplete() {
            sink.complete(); 
        }
    });
});

The idea being to create a single thread that keeps on requesting for messages in a loop and then using an observer pattern like above to do a next() on each message received.

0 Answers
Related