Enforcing keyword-only arguments in decorated functions

Question

I have a class with several methods that require a certain argument be present, but for different reasons.

Typically, the argument will be attached to the instance as an attribute, in which case there is no need for the argument to be passed. However, if the attribute was missing (or None) this argument could be optionally passed as a keyword-only argument:

import functools

class Foo:
    def __init__(self, this_kwarg_default=None):
        self.default = this_kwarg_default
    
    @staticmethod
    def require_this_kwarg(reason):
        def enforced(func):
            @functools.wraps(func)
            def wrapped(self, *args, this_kwarg=None, **kwargs):
                if this_kwarg is None:
                    this_kwarg = self.default
                if this_kwarg is None:
                    raise TypeError(f'You need to pass this kwarg, {reason}!')
                return func(self, *args, this_kwarg=this_kwarg, **kwargs)
        
            return wrapped
        return enforced

    require_this_kwarg = require_this_kwarg.__func__

    @require_this_kwarg('because I said so')
    def foo(self, this_kwarg=None):
        print(f'This kwarg is {str(this_kwarg)}')

Mostly, this gives the desired behavior.

>>> myfoo = Foo(42)
>>> myfoo.foo()
This kwarg is 42
>>> myfoo.foo(this_kwarg=4)
This kwarg is 4
>>> yourfoo = Foo()
>>> yourfoo.foo()
Traceback (most recent call last):
  File "", line 1, in 
  File "dec.py", line 15, in wrapped
    raise TypeError(f'You need to pass this kwarg, {reason}!')
TypeError: You need to pass this kwarg, because I said so!

But if any positional argument is passed, I get some unexpected behavior:

>>> myfoo.foo(4)
Traceback (most recent call last):
  File "", line 1, in 
  File "dec.py", line 16, in wrapped
    return func(self, *args, this_kwarg=this_kwarg, **kwargs)
TypeError: foo() got multiple values for argument 'this_kwarg'

It would make sense, then to define Foo.foo to take this_kwarg as a keyword-only argument:

@require_this_kwarg('because I said so')
def foo(self, *, this_kwarg=None):
    print(f'This kwarg is {str(this_kwarg)}')

However...

>>> myfoo.foo(4)
Traceback (most recent call last):
  File "", line 1, in 
  File "dec.py", line 16, in wrapped
    return func(self, *args, this_kwarg=this_kwarg, **kwargs)
TypeError: foo() takes 1 positional argument but 2 positional arguments (and 1 keyword-only argument) were given

In this case, the desired behavior would be to raise TypeError: foo() takes 0 positional arguments but 1 was given, just as would be expected if no decorator were used.

My hope was that functools.wraps would enforce the call signature of the decorated function. Obviously, though, this is not what wraps does. Is there some way to achieve this?

user_4390160 · Answer

Wow, that was way trickier than I expected. I'd be interested to see if someone comes up with a simpler and cleaner solution, but I think this does what you need?

from inspect import getfullargspec
import functools


class Foo:
    def __init__(self, x_default):
        self.default = x_default

    @staticmethod
    def require_x(reason):
        def enforced(func):
            @functools.wraps(func)
            def wrapped(self, *args, **kwargs):
                argspec = getfullargspec(func)
                while True:
                    if 'x' in kwargs:
                        # it's explicitly there, so it will have a value
                        if kwargs['x'] is None:
                            kwargs['x'] = self.default
                        break
                    elif argspec.varargs is None:
                        # there are no varargs to eat up positional arguments
                        if 'x' in argspec.args[:len(args)+1]:
                            # x will get a value from args, offset by one for self
                            if args[argspec.args.index('x') - 1] is None:
                                args = tuple(a if n != argspec.args.index('x') - 1 else self.default
                                             for n, a in enumerate(args))
                            break
                        elif argspec.defaults is not None and 'x' in argspec.args[-len(argspec.defaults):]:
                            # x will get a value from a default
                            if argspec.defaults[argspec.args[-len(argspec.defaults):].index('x')] is None:
                                kwargs['x'] = self.default
                            break
                    elif 'x' in argspec.kwonlydefaults:
                        if argspec.kwonlydefaults['x'] is None:
                            kwargs['x'] = self.default
                        break
                    raise TypeError(f'{func.__name__} needs a value for x, {reason}.')

                func(self, *args, **kwargs)

            return wrapped

        return enforced

    require_x = require_x.__func__

I don't like production code that needs inspect to work, so I'm still dubious of whether you really need code that does this - there is probably a bit of an anti-pattern in the broader design here. But anything can be done, I suppose.

user_6238395 · Answer

This doesn't really answer the question; but it is a workable solution, illustrates the desired behavior, and is much too long for a comment anyway.

Maybe this is a bad/stupid idea. I'm not sure.

What I want is basically an optional keyword argument:

def foo(self, this_kwarg=None):
    if this_kwarg is None:
        this_kwarg = self.default
    ...

But what if self.default is None? In general, this should be allowed. However, certain methods such as Foo.foo require that this this_kwarg is not None, i.e. they will fail if it is. So basically I am looking for descriptive/informative error handling in a nice 'pythonic' way.

One way around it is to implement Foo.foo like this:

def foo(self, this_kwarg=None):
    if this_kwarg is None:
        this_kwarg = self.default
    if this_kwarg is None:
        raise TypeError('This kwarg cannot be `None`, because I said so!')
    ...

But then I have to add this code to every method that has this requirement. (Suppose I have several other methods Foo.bar, Foo.baz etc., that all cannot work if the argument is None.)

I thought a decorator would be an elegant & pythonic way to implement this without lots of repetitive code. The problem is that Foo.foo, Foo.bar, Foo.baz, etc. all have different call signatures. Besides this_kwarg, some of these methods may have 1 or more (maybe an arbitrary number) of positional and/or keyword arguments.

Maybe, then, the best solution is this:

class Foo:
    def __init__(self, default=None):
        self.default = default
    
    def require_this_kwarg(self, this_kwarg, reason):
        if this_kwarg is None:
            this_kwarg = self.default
        if this_kwarg is None:
            raise TypeError(f'This kwarg cannot be `None`, because {reason}!')
        return this_kwarg

    def foo(self, *args, this_kwarg=None, **kwargs):
        this_kwarg = self.require_this_kwarg(this_kwarg, 'I said so')
        ...

This was actually my original solution. Then I tried to do it with a decorator & ran into the issue I described. I thought the question of whether function call signatures can be enforced inside a decorator was an interesting one.

user_10642305 · Answer

After read your question and comments, my understanding about your question is you are searching value in following order.

If this_kwarg exist, use it.
If #1 failed, use self.default.
If #2 failed, raise error.

This code should work in either position or keyword argument.

import functools

class Foo:
    def __init__(self, this_kwarg_default=None):
        self.default = this_kwarg_default
    
    @staticmethod
    def require_this_kwarg(reason):
        def enforced(func):
            @functools.wraps(func)
            def wrapped(self, this_kwarg=None):
                def _process(k=self.default):
                    if k is None:
                        raise TypeError(f'You need to pass this kwarg, {reason}!')
                    return func(self, k)
                if this_kwarg is None:
                    return _process()
                return _process(this_kwarg)
            return wrapped
        return enforced

    require_this_kwarg = require_this_kwarg.__func__

    @require_this_kwarg('because I said so')
    def foo(self, this_kwarg=None):
        print(f'This kwarg is {str(this_kwarg)}')

The core logic is in code below

                    ...
 12                 def _process(k=self.default):
 13                     if k is None:
 14                         raise TypeError(f'You need to pass this kwarg, {reason}!')
 15                     return func(self, k)
 16                 if this_kwarg is None:
 17                     return _process()
 18                 return _process(this_kwarg)
                    ...

Code logic matches searching order mentioned above:

If this_kwarg isn't None, return func(this_kwarg). (line 18)
If this_kwarg is None, try return func(self.default). (line 17)
If both this_kwarg and self.default are None, raise error. (line 14)

Test

import pytest

@pytest.mark.parametrize("default_val", [None, 1, "this_kwarg_default=1"])
@pytest.mark.parametrize("call_val", [None, 3, "this_kwarg=3"])
def test_foo(default_val, call_val):
    print("parametr are: ", default_val, call_val)
    f = eval(f"Foo({default_val})")
    eval(f"f.foo({call_val})")

Output:

============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.8.5, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
rootdir: /Users/kz2249/tmp/st/tests
collected 9 items                                                                                                                                                                

test_example.py parametr are:  None None
Fparametr are:  1 None
This kwarg is 1

.parametr are:  this_kwarg_default=1 None
This kwarg is 1

.parametr are:  None 3
This kwarg is 3

.parametr are:  1 3
This kwarg is 3

.parametr are:  this_kwarg_default=1 3
This kwarg is 3

.parametr are:  None this_kwarg=3
This kwarg is 3

.parametr are:  1 this_kwarg=3
This kwarg is 3

.parametr are:  this_kwarg_default=1 this_kwarg=3
This kwarg is 3

.

==================================================================================== FAILURES ====================================================================================
______________________________________________________________________________ test_foo[None-None] _______________________________________________________________________________

default_val = None, call_val = None

    @pytest.mark.parametrize("default_val", [None, 1, "this_kwarg_default=1"])
    @pytest.mark.parametrize("call_val", [None, 3, "this_kwarg=3"])
    def test_foo(default_val, call_val):
        print("parametr are: ", default_val, call_val)
        f = eval(f"Foo({default_val})")
>       eval(f"f.foo({call_val})")

test_example.py:36: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
:1: in 
    ???
test_example.py:19: in wrapped
    return wrap()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

k = None

    def wrap(k=self.default):
        if k is None:
>           raise TypeError(f'You need to pass this kwarg, {reason}!')
E           TypeError: You need to pass this kwarg, because I said so!

test_example.py:16: TypeError
============================================================================ short test summary info =============================================================================
FAILED test_example.py::test_foo[None-None] - TypeError: You need to pass this kwarg, because I said so!
========================================================================== 1 failed, 8 passed in 0.10s ===========================================================================

Enforcing keyword-only arguments in decorated functions

3 Answers