Gunicorn reflect changed code dynamically

user_1729802 0
Viewed 582

I am developing a django web application where a user can modify the code of certain classes, in the application itself, through UI using ace editor (think of as gitlab/github where you can change code online). But these classes are ran by django and celery worker at some point.

Once code changes are saved, the changes are not picked by django due to gunicorn but works fine with celery because its different process. (running it locally using runserver works fine and changes are picked by both django and celery).

Is there a way to make gunicorn reflects the changes of certain directory that contain the classes without reloading the whole application? and if reloading is necessary, is there a way to reload gunicorn's workers one-by-one without having any downtime?

the gunicron command:

/usr/local/bin/gunicorn config.wsgi --bind 0.0.0.0:5000 --chdir=/app

The wsgi configuration file:

import os
import sys

from django.core.wsgi import get_wsgi_application

app_path = os.path.abspath(os.path.join(
    os.path.dirname(os.path.abspath(__file__)), os.pardir))
sys.path.append(os.path.join(app_path, 'an_application'))

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.production")

application = get_wsgi_application()
gunicorn django
3 Answers

The reload option is "intended for development". There's no strong wording saying you shouldn't use it in production. The reason you shouldn't use it in production is because people make typos, change in one file, may need several other changes in others, etc etc. So, you can make your site inaccessible and then you don't have a working app to fix it again.

For a dev, that's no problem as you look at the logs/output in your shell and restart it. This is why @Krzysztof's suggestion is the best one. Push the code changes to your repo, make it go through the CI/CD and switch over the pod. If CI fails, then CD won't happen so you're good.

Of course, that's a scope far too large for a Q&A site.

Admin1

Why not save the code in a separate text file or database and the relevant method can simply load the code dynamically as a string and execute it using exec()?

Let say you have a function function1 which can be edited by a user. When the user submits the changes, process the input (separate out the functions so that you know which function has what definition), and save them all individually, like function1, function2 etc., in a database or a text file as strings.

One you need to execute function1, just load its value that you saved and use exec to execute the code.

This way, you won't need to reload gunicorn since all workers will always fetch the updated function definition at run time!

Something in the lines of:


def function1_original():
    # load function definition
    f = open("function1.txt", "r")
    
    # execute the string
    exec(f.read())  # this will just load the function definition
    function1()  # this will execute the user defined function

So the user will define:

def function1():
    # user defined code
    # blah blah
    ...
user_41860080

I was able to solve this by changing the extension of the python scripts to anything but .py

Then I loaded these files using the following function:

from importlib import util
from immportlib.machinary import SourceFileLoader

def load_module(module_name, modele_path):
    module_path = path.join(path.dirname(__file__),  "/path/to/your/files{}.anyextension".format(module_name))
    spec = util.spec_from_loader(module_name,
                                 SourceFileLoader(module_name, module_path))
    module = util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module

In this case, they are not loaded by Gunicorn in RAM and I was able to apply the changes on fly without the need to apply eval or exec functiong.

user_17298020
Related