Type annotations for base and inherited classes - is Generic and TypeVar the right approach?

user_3217306 0
Viewed 1581

Say I have a base class

from typing import List, Optional

class Node:
    def __init__(self, name: str) -> None:
        self.name = name
        self.children: List['Node'] = []
    ...

and a subclass

class PropertiesNode(Node):
    def __init__(self, name: str, properties: List[str], inherit: Optional['PropertiesNode']) -> None:
        Node.__init__(self, name)
        self.properties = set(properties)
        if inherit:
            self.properties.update(inherit.properties)
            self.children = deepcopy(inherit.children)
            for child in self.children:
                child.properties.update(properties)  # ERR: "Node" has no attribute "properties"  [attr-defined]

As you can see, mypy (rightly) flags an error there, as Node.children was explicitly given a type of List[Node].

So I read up on generic types, and it seems to me the solution is to use TypeVars and Generic:

from typing import Generic, List, Optional, TypeVar

N = TypeVar('N', bound='Node')
P = TypeVar('P', bound='PropertiesNode')

class Node(Generic[N]):
    def __init__(self: N, name: str) -> None:
        self.name = name
        self.children: List[N] = []

class PropertiesNode(Node[P]):
    def __init__(self: P, name: str, properties: List[str], inherit: Optional[P]) -> None:
        Node.__init__(self, name)
        self.properties = set(properties)
        if inherit:
            self.properties.update(inherit.properties)
            self.children = deepcopy(inherit.children)
            for child in self.children:
                child.properties.update(properties)

However, now when I instantiate the classes, I get

foo = Node("foo")  # ERR Need type annotation for "foo"  [var-annotated]
bar = PropertiesNode("bar", ["big", "green"], None)  # ERR Need type annotation for "bar"  [var-annotated]

Now, I could silence these by doing

foo: Node = Node("foo")
bar: PropertiesNode = PropertiesNode(...)

but why does that silence it - I'm not giving mypy any new info there? The more I think about, the less Generic seems like the right choice, because the thing is: all instances of Node or PropertiesNode will have self.children that are of exactly the same type as self.

But if I remove the Generic[N] from class Node(Generic[N]):, I end up with the original error again:

class PropertiesNode(Node):
    ...
                child.properties.update(properties)  # ERR "N" has no attribute "properties"  [attr-defined]
python-typing mypy type-hinting python generics
1 Answers

There are two things going on here

1. Generics

Annotating a variable foo: Node, where Node is a generic class, is equivalent to annotating it as foo: Node[typing.Any]. It will silence MyPy on the default settings, but if you choose to use MyPy with some of the stricter flags set to True (which I recommend doing!), you'll find that MyPy still flags this kind of thing as an error.

If you run this in MyPy:

from typing import TypeVar, Generic, List

N = TypeVar('N', bound='Node')

class Node(Generic[N]):
    def __init__(self: N, name: str) -> None:
        self.name = name
        self.children: List[N] = []

foo: Node = Node("foo")
reveal_type(foo)

You'll find that MyPy will come back to you with a message similar to:

Revealed type is "__main__.Node[Any]"

(N.B. reveal_type is a function that MyPy recognises, but that will fail if you try to use it at runtime.)

To get MyPy to flag unparameterised generics as errors, run MyPy with the command-line argument --disallow-any-generics. Doing so will mean MyPy will flag the following errors:

main.py:3: error: Missing type parameters for generic type "Node"
main.py:10: error: Missing type parameters for generic type "Node"

Forcing you to adjust your code to the following:

from typing import TypeVar, Generic, List, Any

N = TypeVar('N', bound='Node[Any]')

class Node(Generic[N]):
    def __init__(self: N, name: str) -> None:
        self.name = name
        self.children: List[N] = []

foo: Node[Any] = Node("foo")

This makes MyPy happy once again, and says the same thing that you said in your original code, but more explicitly.

However...

2. I don't think it's necessary to use Generics in this situation

You don't have to inherit from generic in order to annotate the self argument in __init__ with a TypeVar. Moreover, as you say in your question, inheriting from Generic just doesn't really make sense here, either from the perspective of MyPy or from other humans reading your code. I'd modify your code like so:

from typing import List, Optional, TypeVar, Any
from copy import deepcopy
    
N = TypeVar('N', bound='Node')
P = TypeVar('P', bound='PropertiesNode')

class Node:
    def __init__(self: N, name: str, *args: Any, **kwargs: Any) -> None:
        self.name = name
        self.children: List[N] = []
    
class PropertiesNode(Node):
    def __init__(self: P, name: str, properties: List[str], inherit: Optional[P], *args: Any, **kwargs: Any) -> None:
        super().__init__(name)
        self.properties = set(properties)
        if inherit is not None:
            self.properties.update(inherit.properties)
            self.children: List[P] = deepcopy(inherit.children)
            for child in self.children:
                child.properties.update(properties)

Now we have annotations that will make MyPy happy, even on the strictest settings, and they even make sense to humans as well!

N.B. I changed two other things in your code here:

  1. I added *args, **kwargs parameters to your __init__ methods — as written, they violated the Liskov Substitution Principle. By adding in these parameters, you avoid that issue.
  2. I changed your test from if inherit to if inherit is not None — lots of things can be False-y in python, so it's much safer to test by identity when testing if a value is None or not.
user_139900160
Related