Why does mypy reject my "mixed union" type declaration?

user_953482 0
Viewed 4432

While troubleshooting a semi-related problem in the Python chat, I came upon some behavior in mypy that I don't understand.

from typing import Union, List, Dict

def f(x: Union[
            Dict[str, float],
            Dict[str, str],
            Dict[str, int],
    ]):
        pass

f({"a": 1})     #passes
f({"a": "b"})   #passes
f({"a": 1.0})   #passes

def g(x: Union[
            Dict[str, float],
            Dict[str, Union[str, int]],
    ]):
        pass

g({"a": 1})     #fails
g({"a": "b"})   #fails
g({"a": 1.0})   #passes

def h(x: Dict[str, Union[float, str, int]]):
    pass

h({"a": 1})     #passes
h({"a": "b"})   #passes
h({"a": 1.0})   #passes

When I execute mypy on this script, it only complains about the middle function, g:

C:\Users\Kevin\Desktop>mypy test.py
test.py:20: error: Argument 1 to "g" has incompatible type "Dict[str, int]"; expected "Union[Dict[str, float], Dict[str, Union[str, int]]]"
test.py:20: note: "Dict" is invariant -- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance
test.py:20: note: Consider using "Mapping" instead, which is covariant in the value type
test.py:21: error: Argument 1 to "g" has incompatible type "Dict[str, str]"; expected "Union[Dict[str, float], Dict[str, Union[str, int]]]"
test.py:21: note: "Dict" is invariant -- see http://mypy.readthedocs.io/en/latest/common_issues.html#variance
test.py:21: note: Consider using "Mapping" instead, which is covariant in the value type
Found 2 errors in 1 file (checked 1 source file)

(As the notes imply, replacing Dict with Mapping removes the errors, but let's say for the sake of the question that I must use Dict.)

These errors are surprising to me. As far as I can tell, the type annotations for each function should simplify down to the same group of types: a dict whose keys are strings, and whose values are floats/strings/ints. So why does only g have incompatible types? Is mypy somehow confused by the presence of two Unions?

mypy typing annotations python
2 Answers

This is because Dict is invariant. It should be invariant because it is mutable.

Dict[str, int] is not a subtype of Dict[str, Union[str, int]] (even though int is a subtype of Union[int, str])

What if you are going to do something like this:

d: Dict[str, Union[str, int]]
u: Dict[str, int]
d = u  # Mypy error: Incompatible type
d["Key"] = "value"

Mypy assumes that dictionaries are homogeneous: they will only ever contain one kind of type. In contrast to this, for example, Tuples are meant to contain heterogeneous data: each item is allowed to have a different type.

If you need heterogenous Dict, you could use TypedDict, but only a fixed set of string keys is expected:

from typing import List, TypedDict

Mytype = TypedDict('Mytype', {'x': str, 'a': List[str]})
s: Mytype = {"x": "y", "a": ["b"]}

s['a'].append('c')

NOTE:

Unless you are on Python 3.8 or newer (where TypedDict is available in standard library typing module) you need to install typing_extensions using pip to use TypedDict

user_137826690

The issue is that Union membership is modelled as subtyping, yet Dict keys/values require exact type matches. This is enforced by MyPy for nested types (g), but loosely interpreted for direct substitution ( h).

The Union type is modelled as a (virtual) subtyping relation. That is, str is considered a subtype of Union[str, x].

  • This matches how issubbclass(a, (str, int)) checks whether a is "either str or int" without saying which. We can use a str value in place of a (str, int)-union as long as only the common features are used.

The Dict type is invariant in its key/value types. That is, a key/value must be exactly of the type as declared.

  • For a mutable type such as dict, the key/value are both input d[k] = v and output v = d[k]. Any substitute would have to work with less capable input or provide more capable output – which is not possible, since input and output must have the same capabilities.

In combination, using a Dict[..., Union[str, ...]] requires the value to match the Union[str, ...] exactly – the (virtual) subtype str is not valid. As such, {"a": "b"} is considered as Dict[str, str] which is not a substitute for Dict[str, Union[str, int]].

Since logically a union follows slightly separate behaviour from regular types, MyPy has some separate code paths for Union. This is mostly focused on function signatures and overloading, for which separate union math exists. Thus, certain flat type substitutions allows for more practical Union matching, as is the case for h.

user_53499160
Related