Is there a way to restart a GenServer with different configuration after a abnormal exit on Elixir?

user_1204726 0
Viewed 248

I started to learn Elixir a couple of days ago and I'm really liking it. Especially the idea of using a Supervisor processes as helpers to create fault-tolerant applications.

However, one scenario that I thought would be natural but I couldn't find any reference so far, is a way of restarting a child process (GenServer) with a different configuration upon a failure.

My use case is the following:

I have a process that grabs data from a Database and works with it to some extent. And the database connection is crucial for the process life. I want to be able to use a different database connection (a backup database) if the process exits abnormally because it could not connect to/fetch data from the database a given N number of times (instead of unleashing hell and throwing notifications that the process is dead because of database connectivity)

Is it possible?

As far as I could see, the Supervisor simply restarts the process the same way every time. Maybe using a :simple_one_for_one strategy could suffice somehow, but I don't know exactly how.

erlang-otp elixir
1 Answers

AFAIK, this is not possible with a single option, because one needs to store a state of this process (default vs backup.)

That is relatively easy to implement the desired behavior, though.

First of all, make the supervisor load and maintain the connection pool, e. g. from the config file. Secondary, yes, it’s easier to use :simple_one_for_one restart strategy. Important: make it explicitly trap the exits:

Process.flag :trap_exit, true

And, finally, when :EXIT signal is received, rotate the connection pool, make a cleanup and manually restart the underlying critical process with start_link, passing the currently head connection settings from the connection pool as the argument to it.

A hacky but faster solution would be to store the connection pool in DETS (or any other persistent storage,) read it on initialization and pass to the underlying child, immediately rotating and overriding the list in DETS. That way one might use :one_for_one or any other desired strategy. One should cleanup the storage on each supervisor restart to make the default connection setting to be on top at that moment. I would not recommend this approach.

Update: Apparently, just using long-running Task and handling :DOWN signal from it could suffice in your particular case.

user_20352620
Related