Futures: set result from a different thread

user_5057191 0
Viewed 850

I have an http server that receives some requests in a thread. This http server parses the received data and sends part of the data to a different process.

The response from this process may take some time and is received on a callback in a different thread.

The issue here is that I need part of the response received in the callback function to respond to the initial request I received, so my idea was to use an asyncio.Task for this

My code is as follows:

my_futures = dict()

class HTTPHandler(http.server.BaseHTTPRequestHandle):
    do_POST(self):
        #parse data
        asyncio.create_task(self.send_response())

    async def send_response(self):
        global my_futures
        my_futures[uuid] = asyncio.get_event_loop().create_future()
        send_data_to_processB()
        await my_future
        self.send_response()

# Callback where I receive the result from process B
def on_message(message):
    global my_futures
    my_futures[message['uuid']].set_result(message['result'])

if __name__ == '__main__':
    server = http.server.HTTPServer(LISTEN_ADDR, MyHTTPHandler)
    threading.Thread(target=server.serve_forever).start()
    processB.register_callback(on_message)

The problem with this approach is that the task is not being executed and the callback does not receive results at all.

I also tried changing the do_POST method to use asyncio.run(self.send_response()) instead of asyncio.create_task, using this approach my callback gets the results and sets the result but the coroutine just hangs at await my_future

How can I achieve this task?

python-asyncio coroutine asynchronous python-3.x
2 Answers

To interact with asyncio from outside the event loop thread, use call_soon_threadsafe:

def on_message(message):
    # my_loop must have been initialized from the main thread
    my_loop.call_soon_threadsafe(
        my_futures[message['uuid']].set_result, message['result'])

Instead of create_task, call asyncio.run_coroutine_threadsafe.

However, this is not the only problem with your code. You are never running the event loop, so none of this has a chance to execute. You need to have asyncio.run or equivalent somewhere in your code, typically at top-level.

Since HTTPServer is not written for asyncio, you cannot call asyncio functions from do_POST. Instead, look into using aiohttp.

user_16008980

Same as above, but without the need to save a reference to the event loop

future = my_futures[message['uuid']]
result = message['result']
future.get_loop().call_soon_threadsafe(future.set_result, result)
user_85231170
Related