Unsure how asyncio works and how I return information from a continuously running async folder watchdog function

user_12971025 0
Viewed 479

I am working on a tool that watches up to 3 folders for changes.
If a change occurs its passes the path into a function.
This function is supposed to gather information and return them in a way that I can for example use it to be displayed on a GUI. The code is as follows.

# Contains code for different checks on the output folder of the AOIs
from export_read import aoi_output_read
import asyncio
from watchgod import awatch
import config_ini


async def __run_export_read(path):
    # On every trigger of the function read a .ini with config settings
    # this .ini gets changed from the outside (by hand or with a tool)
    # This way we can modify the actions that happen on every read
    # The config file contains the name of the functions in export_read. If the value of the key is 1 it uses getattr to run that function and return that value
    # For example it runs "list_of_errors" if that key is set to 1 in the ini file and returns the result of that
    config_cls = config_ini.export_read_config()
    active_tools = config_cls.read_ini()
    aoi_output_read_cls = aoi_output_read(path)
    if not aoi_output_read_cls.is_valid():  # checks if file is valid. Returns if not
        return
    for tool in active_tools:
        try:
            method = getattr(aoi_output_read_cls, tool)  # This gets the method name out of the loaded config file string
        except AttributeError:
            print("Class `{}` does not implement `{}`".format(aoi_output_read_cls.__class__.__name__, tool))
        yield method()  # "method" runs the function that got read from the ini file
        # todo: I need to figure out how to keep returning data from this function as long as the async loop from the watchdog is running


async def __watchdog(path):
    async for changes in awatch(path):
        for items in changes:
            if items[0] == 1 or items[0] == 2:  # when File is created or changed
                async for item in __run_export_read(items[1]):
                    yield item  # This doesnt work as run_until_complete(__watchdog(path)) needs a different output. Which I do not understand

The yield inside __watchdog() does not work because I cant pass a list to run_until_complete. But this is as far as my knowledge goes.

def folder_watchdog(path):

    loop = asyncio.get_event_loop()
    loop.run_until_complete(__watchdog(path))

My main file.

import analysis


def main():
    path = r"I:\Data Export\5"
    print(analysis.folder_watchdog(path))


main()

Currently I just want to be able to use the print statement in main to know that that data gets passed through all of this code.

What can I do to get the data I need without stopping the execution. Is there a completley different way of doing it?

python-asyncio python-3.x python
1 Answers

Solved this with the following(Solution from user4815162342)

analysis.py:

def iter_over_async(ait, loop):
    ait = ait.__aiter__()

    async def get_next():
        try:
            obj = await ait.__anext__()
            return False, obj
        except StopAsyncIteration:
            return True, None
    while True:
        done, obj = loop.run_until_complete(get_next())
        if done:
            break
        yield obj


def folder_watchdog(path):
    while True:
        loop = asyncio.get_event_loop()
        for items in iter_over_async(__watchdog(path), loop):
            yield items

Now in my main file print works like this:

def main():
    path = r"I:\AOI Data Export\L5"
    for stuff in analysis.folder_watchdog(path):
        print(stuff)

main()
user_129710250
Related