Knowing full well that typing.cast should be reserved for the most specialest of special occasions, is this a time to cast away anyway??
Adding custom attributes to function objects is a pretty neat trick (e.g. functools.lru_cache), and, thank the good Lord for Python, it's trivial to do so with setattr. Unfortunately, it's not so trivial to have those shiny new attributes show up in autocomplete lookaheads or pass muster with static type checkers and linters (without implicitly or explicitly resorting to Any). For example,
def badfoo():
"""not gonna work"""
print("oh no")
setattr(badfoo, "new_attr", "here it is")
badfoo.new_attr
works at runtime but the new attribute doesn't show up in autocomplete nor is it recognized by mypy or pylint:
pylint: testvcd.py:38:0: E1101: Function 'badfoo' has no 'new_attr' member (no-member)
mypy: testvcd.py:38: error: "Callable[[], Any]" has no attribute "new_attr"
Adding the attribute using an untyped decorator prevents the pylint and mypy errors, but it doesn't solve the autocomplete issue or allow for type checks on the appended attributes(s).
EDIT
Thanks, @Markus Unterwaditzer, for recommending a much more concise method for obtaining the desired results as well as a much more precise statement of the issue I'm attempting to address, namely, adding a statically discoverable, type-checkable custom attribute to a function. @Markus' recommended method (with some slight additions to preserve the decorated function's signature) defines the decorator as:
"""attribute_decorator.py: define a `with_attribute` decorator"""
from typing import TypeVar, Generic, Callable, ParamSpec
T = TypeVar("T")
T_ATTR = TypeVar("T_ATTR")
P = ParamSpec("P")
class _FuncWithAttr(Generic[P, T, T_ATTR]):
new_attr: T_ATTR
def __call__(self: Callable[P, T], *args: P.args, **kwargs: P.kwargs) -> T:
...
def with_attribute(value: T_ATTR) -> Callable[[Callable[P, T]], _FuncWithAttr[P, T, T_ATTR]]:
def inner(f: Callable[P, T]) -> _FuncWithAttr[P, T, T_ATTR]:
f.new_attr = value # type: ignore
return f # type: ignore
return inner
and usage:
from attribute_decorator import with_attribute
@with_attribute({"my": "dict"})
def foo(some: int, args: int) -> int:
return some + args
# typechecks!
x = {"your": "dict"}
x |= foo.new_attr
assert foo(1, 2) == 3
As shown, Markus' method still requires some "# type: ignore" directives. I'm still hoping someone might come along and demonstrate a method that pulls this off without casts (used in my original solution below) or ignores (used in Markus' solution above), so if you know any mypy devs who are both bored and benevolent, please send them my way... XD
Stop reading here unless you're interested in my original, more confusing example ;)
As a chronic circumventionalist, I set about fumbling my way around those limitations in what seemed a semi-coherent manner and, lo and behold, I managed to crack the code... in the decorator use case... kinda... **
**I'm relying on typing.cast to make it work. I wrote the following "RollingDefaults" decorator as a (slightly) practical example that demonstrates the typing mechanism. I'm sure it would fail at its stated purpose for lots of use cases, but that's beside the point. The question is whether or not I could achieve the same discovery/static checker benefits without the reliance on typing.cast. Note that these benefits extend to enforcement of type checks on any and all appended attributes as well as the args/kwargs passed to the wrapped function, so cast seems like a fairly effective solution, at least on the surface.
rolling_defaults.py
"""use the most recently passed parameter values as the defaults for
subsequent calls to the wrapped function and provide visibility to the
current (rolling) and original default parameter values"""
from collections.abc import Callable
from functools import wraps
from typing import Any, Generic, ParamSpec, TypeVar, cast
T = TypeVar("T")
P = ParamSpec("P")
class _RollingDefaultFunc(Generic[P, T]):
rolling_defaults: dict[str, Any]
original_defaults: dict[str, Any]
def __call__(self: Callable[P, T], *args: P.args, **kwargs: P.kwargs) -> T:
...
def reset_defaults(self):
"""return to original defaults"""
class RollingDefaults:
"""add "rolling_defaults", "original_defaults", and "reset_defaults"
attributes to wrapped function
"""
def __init__(self):
self.func: Callable
self._defaults: dict[str, Any]
self._rolling_defaults: dict[str, Any]
def __call__(self, func: Callable[P, T]) -> _RollingDefaultFunc[P, T]:
self.func = func
self._rolling_defaults = {
k: v() if isinstance(v, type) else None
for k, v in func.__annotations__.items()
if k != "return"
}
true_defaults = [] if func.__defaults__ is None else list(func.__defaults__)
if func.__kwdefaults__ is not None:
true_defaults.extend(list(func.__kwdefaults__.values()))
type_defaults = list(self._rolling_defaults.values())
for diff_idx in reversed(range(len(type_defaults) - len(true_defaults))):
true_defaults.insert(0, type_defaults[diff_idx])
self._defaults = dict(zip(self._rolling_defaults, true_defaults))
self._rolling_defaults |= self._defaults
@wraps(func)
def rolling_wrapper(*args: P.args, **kwargs: P.kwargs) -> T:
return self._wrapper(*args, **kwargs)
setattr(rolling_wrapper, "rolling_defaults", self.rolling_defaults)
setattr(rolling_wrapper, "original_defaults", self.original_defaults)
setattr(rolling_wrapper, "reset_defaults", self.reset_defaults)
#
# Is cast a good option here?
#
return cast(_RollingDefaultFunc[P, T], rolling_wrapper)
#
# Is there a better way that doesn't use the cast above?
#
@property
def rolling_defaults(self) -> dict[str, Any]:
"""parameter defaults for the next call"""
return self._rolling_defaults
@property
def original_defaults(self) -> dict[str, Any]:
"""original defaults"""
return self._defaults
def reset_defaults(self):
"""reset to originals"""
self._rolling_defaults |= self._defaults
def _wrapper(self, *args, **kwargs) -> T:
self._rolling_defaults |= dict(zip(self._defaults, args))
self._rolling_defaults |= kwargs
return self.func(**self._rolling_defaults)
testroll.py
from rolling_defaults import RollingDefaults
@RollingDefaults()
def foo(bar: int = 1, *, baz: str = "test") -> str:
return f"{bar=}; {baz=}"
if __name__ == "__main__":
print(foo())
print(foo.rolling_defaults)
print(foo(11, baz="passed"))
print(foo.rolling_defaults)
print(foo())
foo.reset_defaults()
print(foo.rolling_defaults)
print(foo())
Result
Autocomplete works for the new attributes and displays the correct call signature, and pylint / mypy no longer raise attribute errors but do raise appropriate errors on bad assignments to the appended attributes and / or bad arg values passed in the call to foo. Output:
>python testroll.py
bar=1; baz='test'
{'bar': 1, 'baz': 'test'}
bar=11; baz='passed'
{'bar': 11, 'baz': 'passed'}
bar=11; baz='passed'
{'bar': 1, 'baz': 'test'}
bar=1; baz='test'