How can I run an async function from a django view to run in the background while I process requests in the main loop?

user_15442678 0
Viewed 1930

As I said in the title, I want a task to run in the background processing a large database query while I handle requests from my frontend. Can I even do that with async/asyncio? I was told it was possible...

For the purpose of context, I would like to do something like the following. Note also that I don't really need the function to tell me when it's done (though I'd sure like to know if it's possible) since I just check if the .json file was finally written.


def post_data_view(request):
    if request.method == 'POST':
        ...
        do_long_query_in_the_background(some_data)
        return HttpResponse('Received. Ask me in a while if I finished.')

def is_it_done_view(request):
    if request.method == 'GET':
        data = find_json()
        if data:
            return JsonResponse(data)
        else:
            return HttpResponse('Not yet dude...')

async def do_long_query_in_the_background(data):
    # do some long processing...
    # dump the result to a result.json
    return

I was told this was possible with async but I really find it hard to understand. For context, I tried to simplify this a lot, and even then I found I didn't quite understand what was happening:


async def f():
    while True:
        print(0)
        await asyncio.sleep(2)

asyncio.create_task(f())

even this code I tried fails sys:1: RuntimeWarning: coroutine 'f' was never awaited, but it does work on console and I do not understand why that is.

I also wondered if this is at all possible and safe to do with threading maybe?

I'm extremely frustrated with this because the general solution suggested in other threads seems to be to just use celery but it really feels like an overkill for a not so complex problem.

asynchronous multithreading django python
2 Answers
async def f():
    while True:
        print(0)
        await asyncio.sleep(2)

asyncio.run(f())

the run method from asyncio sets up your event loop and creates the Task object, schedules it to run and waits for it to complete before executing any other code.

While appearing quite simple, async programming requires a very different approach to programming as things can happen in any order and you have to think very carefully about what order is important for functions to complete.

For your use case though, you might be able to use threading. You might be able to create a new thread any have it running in the background. There is a performance penalty due to switching between threads but your user may have a better experience if most of the processing is done server side.

#this runs forever as your loop never terminates
from threading import Thread
from time import sleep

def f():
    while True:
        print(0)
        await asyncio.sleep(2)

def main():
     print('starting new thread...')
     t = Thread(target=f)
     t.start()
     print('continuing other tasks...')
     sleep(5)
     print('still more things to do...')
user_104511260
  1. You need to define your view function with aync . In your situation it should be: async def post_data_view(request):
  2. In your async view function: Firstly need to get loop. Then create a task in the loop. It should be:
    async def post_data_view(request):
        if request.method == 'POST':
        ....
        loop = asyncio.get_event_loop()
        loop.create_task(do_long_query_in_the_background(some_data))
        return HttpResponse('Received. Ask me in a while if I finished.')
user_103522870
Related