Does NestJs support timeout increases for RabbitMQ on the client side?

Viewed 712

In a nutshell, I'm trying to send across a large file (~250-300Mb) by converting the buffer into base64, and having it sent via a producer.

The microservices are run locally for this use case. If I run RabbitMQ locally, it's great, but the consumer keeps having connection resets when I try to have it hosted on the cloud (EC2/EKS clusters). Here's what the logs look like for RabbitMQ.

{inet_error,timeout}
2021-07-04 03:09:23.712 [info] <0.5074.0> Closing all channels from connection 'yy.yy.yy.yy:yy -> xx.xx.xx.xx:xx' because it has been closed
2021-07-04 03:10:08.714 [warning] <0.5081.0> closing AMQP connection <0.5081.0> (yy.yy.yy.yy:yy -> xx.xx.xx.xx:xx, vhost: '/', user: 'John Doe'):
client unexpectedly closed TCP connection

I've checked out the docs (https://docs.nestjs.com/microservices/rabbitmq), and the options for queueOptions & socketOptions - it doesn't look like there's a way to increase timeout options on the client side. I'm using amqplib and amqp-connection-manager to get this gig going.

Here's what the consumer declaration looks like.

async function bootstrap() {
  const app = await NestFactory.createMicroservice(ConsumerModule, {
    transport: Transport.RMQ,
    options: {
      urls: [
        'amqp://xxxx:yyyy@genericdomain:port/'
      ],
      queue: 'x',
      noAck: true,
      prefetchCount: 0,
      queueOptions: {
        durable: true,
      } 
    }
  });
  await app.listenAsync();
}

This setup works for smaller files though, so chunking the data is one option I was toying with, even though it'd involve more work to set up multiple consumers (I know it's not recommended to send large payloads on RabbitMQ, though)

I've edited the configuration to look like this, and it doesn't look like it helps. I've been looking at the config on https://www.rabbitmq.com/configure.html#config-file-formats.

vm_memory_high_watermark.relative = 0.8
vm_memory_calculation_strategy = rss
vm_memory_high_watermark_paging_ratio = 0.5
disk_free_limit.absolute = 10GB
channel_max = 2047
heartbeat = 0
max_message_size = 526870912
handshake_timeout = 50000000
frame_max = 20MB
initial_frame_max = 20MB

Have any of you had luck with this?

0 Answers
Related