I use delayed job with the ActiveRecord backend using mysql on a distributed system. Whenever I restart the mysql database, the workers simply stop. According to the log, each worker attempts to reconnect 8 times with 5 second intervals and then quits with a FATAL -- : MySQL client is not connected message.
This means that each time I restart the database, I have to also restart all my workers.
Is there a way to increase the number of reconnection attempts?
In the job:
class MyJob < ActiveJob::Base
retry_on(YourMySQLException, wait: 10.seconds, attempts: 10)
end
If you want this retrying behaviour to be the default for all your jobs, consider putting this line in ApplicationJob, and have all your jobs inherit from it.
The official documentation has different defaults than the ones you inferred from the logs, but I guess it doesn't matter => https://edgeapi.rubyonrails.org/classes/ActiveJob/Exceptions/ClassMethods.html#method-i-retry_on
Nevertheless, another question might be: why do you need to restart mysql? This should probably not happen in a production environment...
