How to delete a redis stream after success processing

user_8282446 0
Viewed 3758

I use redis as transport in messenger, I thought that after processing a flow the deletion was automatic but alas not. I do not know how to delete a repeat stream when the processing has been carried out with success.

I use symfony 4.4.latest and redis server 6.0

Thanks

symfony-messenger symfony4 redis
3 Answers

The way to do it is by using XTRIM command.

You can call you process couple of messages you trim the stream to retain only the messages that were not processed. By, calling XLEN you can get the stream size and if you subtract the amount of messages you processed you should be left with the right argument for the XTRIM.

user_5934250

Simple steps:

  1. Read a stream message and keep its ID.
  2. When you are done processing the message use the XDEL command to delete specific entries from the redis stream.

BTW redis streams are not supposed to be used like this, better use pub/sub functionality as @Dudo mentioned. This is a good introduction to redis streams: https://redis.io/topics/streams-intro

user_2713590

Want to add here that in Symfony 5 ?delete_after_ack=true was added.

MESSENGER_TRANSPORT_DSN=redis://localhost:6379/messages?delete_after_ack=true

Why the default in Symfony 5 is false. The default in Symfony 6 is true so Symfony 6 will automatically remove acked message.

See also: https://symfony.com/doc/current/messenger.html#redis-transport

There are also other parameters like delete_after_reject or stream_max_entries which trims the stream. Keep in mind stream_max_entries is trim messages aways which also are not processed. So it should be a high enough value.

user_23061020
Related