iter() alternative for awaitables?

user_10909029 0
Viewed 486

Usually in IO operations we use iter() to read up to sentinel value:

from sys import stdout

with open(r"Z:\github\StackOverFlow\temp.json", "r") as fp:
    for chunk in iter(lambda :fp.read(64), ""):
        stdout.write(chunk)

But is there alternative to iter() for awaitable, i.e. asyncio.Queue.get()?

for val in iter(lambda: await queue.get(), sentinel):
    queue.task_done()
    print(val)

Surely this won't work as while it requires callable, await can't be called inside non-async functions.

Situation does not allow queue.get_nowait() as queue is empty for most of time.

Simple fix would be using while loop:

while True:
    if (val := await queue.get()) is None:
        break
    queue.task_done()
    print(val)

But I'm afraid if this harms readability and clarity.

python-asyncio python
2 Answers

You can improve what you used with the while loop to integrate the walrus into the while conditional:

while (val := await queue.get()) is not None:
    queue.task_done()
    print(val)

which gets you to equivalent brevity of your desired result, and isn't particularly ugly relative to your desired two-arg iter solution (two-arg iter being a relatively ugly thing to use in the first place).

user_3646960

PEP 525 that introduced asynchronous generators in Python 3.6 actually proposes built-in aiter and anext in Python 3.7. And we know that that didn't happen. Looking at the discussion in corresponding BPO-31861, that still may happen but not earlier than Python 3.10.

But there's the CPython BPO PR with early implementation. There's still discussion whether it should be implemented in Python or C, live in operator module or builtins, but it seems the Python version can already be employed. Including the code verbatim as of 908227bb:

from collections.abc import AsyncIterable, AsyncIterator


_NOT_PROVIDED = object()  # sentinel object to detect when a kwarg was not given


def aiter(obj, sentinel=_NOT_PROVIDED):
    """aiter(async_iterable) -> async_iterator
    aiter(async_callable, sentinel) -> async_iterator
    Like the iter() builtin but for async iterables and callables.
    """
    if sentinel is _NOT_PROVIDED:
        if not isinstance(obj, AsyncIterable):
            raise TypeError(f'aiter expected an AsyncIterable, got {type(obj)}')
        if isinstance(obj, AsyncIterator):
            return obj
        return (i async for i in obj)

    if not callable(obj):
        raise TypeError(f'aiter expected an async callable, got {type(obj)}')

    async def ait():
        while True:
            value = await obj()
            if value == sentinel:
                break
            yield value

    return ait()
user_20720350
Related