How to type mutable default arguments

user_5049813 0
Viewed 1090

The way to deal with mutable default arguments in Python is to set them to None.

For example:

def foo(bar=None):
    bar = [] if bar is None else bar
    return sorted(bar)

If I type in the function definition, then the only type for bar says that bar is Optional when, clearly, it is not Optional by the time I expect to run that sorted function on it:

def foo(bar: Optional[List[int]]=None):
    bar = [] if bar is None else bar
    return sorted(bar) # bar cannot be `None` here

So then should I cast?

def foo(bar: Optional[List[int]]=None):
    bar = [] if bar is None else bar
    bar = cast(List[int], bar) # make it explicit that `bar` cannot be `None`
    return sorted(bar)

Should I just hope that whoever reads through the function sees the standard pattern of dealing with default mutable arguments and understands that for the rest of the function, the argument should not be Optional?

What's the best way to handle this?

EDIT: To clarify, the user of this function should be able to call foo as foo() and foo(None) and foo(bar=None). (I don't think it makes sense to have it any other way.)

EDIT #2: Mypy will run with no errors if you never type bar as Optional and instead only type it as List[int], despite the default value being None. However, this is highly not recommended because this behavior may change in the future, and it also implicitly types the parameter as Optional. (See this for details.)

default-arguments mutable typing python
5 Answers

None is not the only sentinel available. You can choose your own list value to use as a sentinel, replacing it (rather than None) with a new empty list at run time.

_sentinel = []

def foo(bar: List[int]=_sentinel):
    bar = [] if bar is _sentinel else bar
    return sorted(bar)

As long as no one calls foo using _sentinel as an explicit argument, bar will always get a fresh empty list. In a call like foo([]), bar is _sentinel will be false: the two empty lists are not the same object, as the mutability of lists means that you cannot have a single empty list that always gets referenced by [].

user_11268410

Why not just cut out the cast when you shadow bar:

def foo(bar: Optional[List[int]]=None):
    bar : List[int] = [] if bar is None else bar
    return sorted(bar)
user_116598810

I'm not sure what's the issue here, since using Optional[List[int]] as the type is perfectly fine in mypy: https://mypy-play.net/?mypy=latest&python=3.9&gist=2ee728ee903cbd0adea144ce66efe3ab

In your case, when mypy sees bar = [] if bar is None else bar, it is smart enough to realize that bar cannot be None beyond this point, and thus narrow the type to List[int]. Read more about type narrowing in mypy here: https://mypy.readthedocs.io/en/stable/kinds_of_types.html?highlight=narrow#union-types

Here's some other examples of type narrowing:

from typing import *

a: Optional[int]
assert a is not None
reveal_type(a)  # builtins.int

b: Union[int, float, str]
if isinstance(b, int):
    reveal_type(b)  # builtins.int
else:
    reveal_type(b)  # Union[builtins.float, builtins.str]
user_49092280

If you add a default value to a parameter in a function than yes it is optional. Also in your code you're allowing the caller to supply nothing to the function and it still works cause it just creates an empty list.

Also in python even with typing it doesn't enforce the type. That's why its called "type hinting" in the documentation.

So if you want to allow callers to call the function with no arguments and it just sort an empty list then your code here is proper.

def foo(bar=None):
    bar = [] if bar is None else bar
    return sorted(bar)

But if you never want a caller to NOT supply something even None, then you should change you function signature.

def foo(bar: List[int]):
    bar = [] if bar is None else bar
    return sorted(bar)

Now type hinting detects that bar is a List[int]. You have to pass something to foo. So you can do foo(None) which is why you need the None check but now foo() is invalid and throws an error.

If you do want them to not pass in anything then just do this and type hinting still works.

def foo(bar: List[int]=None):
   bar = [] if bar is None else bar
   return sorted(bar)
user_92704880

How about using Sequence[int]? It will be type-checked that the argument (including its default) is almost readonly.

from collections.abc import Sequence

def foo(bar: Sequence[int] = []) -> list[int]:
    return sorted(bar)

Still bar can be mutated as below, but the risk would not be high. (_sentinel: list[int] = [] can be mutated, too.)

def foo(bar: Sequence[int] = []) -> list[int]:
    if isinstance(bar, list):
        # reveal_type(bar)  # => Revealed type is "builtins.list[Any]"
        bar.append(0)
    return sorted(bar)
user_106888980
Related