Python Typing: Copy `**kwargs` from on function to another

user_3813064 0
Viewed 172

It is common pattern in Python extend functions and use **kwargs to pass all keyword arguments to the extended function.

i.e. take

class A:
    def bar(self, *, a: int, b: str, c: float) -> str:
       return f"{a}_{b}_{c}"
   

class B:
    def bar(self, **kwargs):
        return f"NEW_{super().bar(**kwargs)}"


def base_function(*, a: int, b: str, c: float) -> str:
    return f"{a}_{b}_{c}"


def extension(**kwargs):
    return f"NEW_{super().bar(**kwargs)}"

Now calling extension(no_existing="a") would lead to a TypeError, that could be detected by static type checkers.

How can I annotate my extension in order to detect this problem before I run my code?

This annotation would be also helpful for IDE's to give me the correct suggestions for extension.

python-typing python
2 Answers

PEP 612 introduced the ParamSpec (see Documentation) Type.

We can exploit this to generate a decorator that tells our type checker, that the decorated functions has the same arguments as the given function:

from typing import Callable, ParamSpec, TypeVar, cast, Any, Type

# Our test function
def source_func(*, foo: str, bar: int) -> str:
    return f"{foo}_{bar}"

# Define some specification, see documentation
P = ParamSpec("P")
T = TypeVar("T")

# For a help about decorator with parameters see 
# https://stackoverflow.com/questions/5929107/decorators-with-parameters
def copy_kwargs(kwargs_call: Callable[P, Any], target: Type[T]) -> Callable[[Callable], Callable[P, T]]:
    """Decorator does nothing but returning the casted original function"""
    def return_func(func: Callable[..., T]) -> Callable[P, T]:
        return cast(Callable[P, T], func)

    return return_func


@copy_kwargs(source_func, float)
def kwargs_test(**kwargs) -> float:
    print(source_func(**kwargs))
    return 1.2

# define some expected return values
okay: float
broken_kwargs: float
broken_return: str

okay = kwargs_test(foo="a", bar=1)
broken_kwargs = kwargs_test(foo=1, bar="2")
broken_return = kwargs_test(foo="a", bar=1)

Checking this file with pyre gives the correct warnings:

ƛ Found 3 type errors!
src/kwargs.py:30:28 Incompatible parameter type [6]: In anonymous call, for 1st parameter `foo` expected `str` but got `int`.
src/kwargs.py:30:35 Incompatible parameter type [6]: In anonymous call, for 2nd parameter `bar` expected `int` but got `str`.
src/kwargs.py:31:0 Incompatible variable type [9]: broken_return is declared to have type `str` but is used as type `float`.

MyPy just recently (7th April 2022) merged a first implementation for ParamSpec that I did not check yet.

According to the related typedshed Issue, PyCharm should support ParamSpec but did not correctly detect the copied **kwargs but complained that okay = kwargs_test(foo="a", bar=1) would have invalid arguments.

Related Issues:

user_38130640

Based on @kound answer.

To remain DRY, we can do the same without re-declaring return type. Type variable T will be deduced later (not when copy_kwargs is called, but when its returned function is), but it doesn't affect further type checking.

from typing import Callable, ParamSpec, TypeVar, cast, Any

# Our test function
def source_func(*, foo: str, bar: int) -> str:
    return f"{foo}_{bar}"

# Define some specification, see documentation
P = ParamSpec("P")
T = TypeVar("T")

# For a help about decorator with parameters see 
# https://stackoverflow.com/questions/5929107/decorators-with-parameters
def copy_kwargs(kwargs_call: Callable[P, Any]) -> Callable[[Callable[..., T]], Callable[P, T]]:
    """Decorator does nothing but returning the casted original function"""
    def return_func(func: Callable[..., T]) -> Callable[P, T]:
        return cast(Callable[P, T], func)

    return return_func


@copy_kwargs(source_func)
def kwargs_test(**kwargs) -> float:
    print(source_func(**kwargs))
    return 1.2

reveal_type(kwargs_test(foo="a", bar=1))
reveal_type(kwargs_test(foo=1, bar="2"))

And here's mypy playground link to look at this in action.

user_144011600
Related