Is there a way to run pandas run_sql_query asynchronously?

Viewed 390

I am reading data from a Postgres DB with n queries. Each of these queries may take a considerable amount of time to execute on the DB so I would like to execute them asynchronously.

I am trying to accomplish this via async module but with no luck

class Test:

    async def collect_items(self, conn_):
        await pd.read_sql_query(items_query_string, conn_)

    async def test_items(self, conn_):
        self.item_df = self.collect_items(conn_)

    async def collect_stockout(self, conn_):
        await pd.read_sql_query(stockout_query_string), conn_)

    async def test_stockout(self, conn_):
        self.so_df = self.collect_stockout(conn_)

    async def _collect_all_async(self, conn_):
        await asyncio.gather(self.test_items(conn_), self.test_stockout(conn_))

    def collect_all(self, conn_):
        asyncio.run(self._collect_all_async(conn_))

However when I call collect_all I would like to wait for all the queries to complete and this is not the case since it returns instantly without properly setting self.item_df and self.so_df

Why is that the case?

0 Answers
Related