Can the * (unpacking) operator be typed in Python? Or any other variadic args function such that all variadic types are in the result type?

user_488191 0
Viewed 602

Working with type stubs, I'm wondering if it's possible to express a type in Python that allows you to type this correctly for any number of arguments:

def test(*args):
  return args

At first glance, I came with:

T = TypeVar('T')
def test(*args: T) -> Tuple[T, ...]:
  return args

But this of course will only type correctly the first T.

Is the only possible way to write the overrides for all arities?

T1 = TypeVar('T1')
T2 = TypeVar('T2')
T3 = TypeVar('T3')
T4 = TypeVar('T4')

@overload
def test(arg1: T1) -> Tuple[T1]: ...
@overload
def test(arg1: T1, arg2: T2) -> Tuple[T1, T2]: ...
@overload
def test(arg1: T1, arg2: T2, arg3: T3) -> Tuple[T1, T2, T3]: ...
@overload
def test(arg1: T1, arg2: T2, arg3: T3, arg4: T4) -> Tuple[T1, T2, T3, T4]: ...
# etc
def test(*args: Any) -> Tuple[Any, ...]:
  return args

This is not complete either, since it does not carry enough type information to type something like:

x: Tuple[int, int, str] = test(*[1, 2, "4"])
pylance python-typing mypy python
2 Answers

TLDR: @overload is currently the only viable way to annotate some level of variance. PEP 646 -- Variadic Generics will enable annotating variadics in Python 3.11 or via typing_extensions.

*args as a Type Variable Tuple

If *args is annotated as a type variable tuple, however, the types of the individual arguments become the types in the type variable tuple:

Ts = TypeVarTuple('Ts')

def args_to_tuple(*args: *Ts) -> Tuple[*Ts]: ...

args_to_tuple(1, 'a')  # Inferred type is Tuple[int, str]

Note that older versions may not support the *TS syntax and need typing.Unpack instead, which is also available via typing_extensions.

The correct approach to annotate *args is to decide on some level of supported "variance length", and explicitly type this using @overload. Notably, explicit parameters must be positional only – they must be either __anonymous or positional, /. A final catch-all variadic @overload handles the case of more arguments.

from typing import TypeVar, Tuple, Any, overload

T = TypeVar('T')
T1 = TypeVar('T1')
T2 = TypeVar('T2')
T3 = TypeVar('T3')

# positional parameters via `, /` – Python 3.8+ only
@overload
def test(arg1: T1, /) -> Tuple[T1]: ...
# positional parameters via double underscore prefix
@overload
def test(__arg1: T1, __arg2: T2) -> Tuple[T1, T2]: ...
@overload
def test(__arg1: T1, __arg2: T2, __arg3: T3) -> Tuple[T1, T2, T3]: ...
# etc
...
# catch all variadic signature for all other cases
@overload
def test(*args: T) -> Tuple[T, ...]: ...

# implementation can use Any to simplify matching all overloads
def test(*args: Any) -> Tuple[Any, ...]:
  return args

reveal_type(test(1, 2, "three"))     # note: Revealed type is 'Tuple[builtins.int*, builtins.int*, builtins.str*]'
reveal_type(test(1, 2, "three", 4))  # note: Revealed type is 'builtins.tuple[builtins.object*]'
reveal_type(test(1, 2, 3, 4))        # note: Revealed type is 'builtins.tuple[builtins.int*]'

It is worth noting that while packing into variadic parameters can be typed, unpacking arguments generally cannot: Any container but tuple is arbitrary length – for example List[int] = [1, 2, 3] – and thus has no exact type information for its elements.

# unpack tuple of int, int
reveal_type(test(*(1, 2,)))  # note: Revealed type is 'Tuple[builtins.int*, builtins.int*]'
# unpack list of some ints
reveal_type(test(*[1, 2,]))  # note: Revealed type is 'builtins.tuple[builtins.int*]'
user_53499160

This can solved through TypeVarTuple from PEP646, implemented in Python 3.11 or forward compat module typing-extensions.

See this analog example:

from __future__ import annotations

from typing import Any, Callable, Generic
from typing_extensions import TypeVarTuple, Unpack  # typing_extensions only needed for Python < 3.11

Ts = TypeVarTuple("Ts")

class Signal(Generic[Unpack[Ts]]):
    def add_callback(self, func: Callable[[Unpack[Ts]], Any]) -> None:
        ...

    def emit(self, *args: Unpack[Ts]) -> None:
        ...

def callback(a: int, b: str) -> None:
    ...

def callback_bad(a: str) -> None:
    ...

sig: Signal[int, str] = Signal()
sig.add_callback(callback)  # Good lint
sig.add_callback(callback_bad)  # Bad lint

sig.emit(1223, "foo")  # Good lint
sig.emit("bar")  # Bad lint

Also see Dynamic TypeVar for a sequence of types for a solution.

user_23878350
Related