Mypy method definition is incompatible with definition in base class

Viewed 1459

I have a parent class that does something and two child classes that wrap a method in orthogonal ways. When I try to combine the two child classes, mypy complains with the following error.

Definition of "run" in base class "Foo" is incompatible with definition in base class "Bar"

How are the two classes "incompatible" and how do I modify my code to appease mypy? Can I do it without making

class Parent:
    def run(self, a, b):
        pass


class Foo(Parent):
    def run(self, a, b, foo=None, **kwargs):
        print('foo', foo)
        super().run(a, b, **kwargs)


class Bar(Parent):
    def run(self, a, b, bar=None, **kwargs):
        print('bar', bar)
        super().run(a, b, **kwargs)


class C(Foo, Bar):
    pass
3 Answers

Mypy complains only because you are trying to inherit C from Foo and Bar, which have the same method run, but with different arguments. To make Mypy happy you have to either make your run functions in Foo and Bar classes the same (match same function signature), or do not inherit class C from both of them.

One simple fix is to make foo and bar keyword-only arguments:

class Parent:
    def run(self, a, b):
        pass


class Foo(Parent):
    def run(self, a, b, *, foo=None, **kwargs):
        print('foo', foo)
        super().run(a, b,  **kwargs)


class Bar(Parent):
    def run(self, a, b, *, bar=None, **kwargs):
        print('bar', bar)
        super().run(a, b, **kwargs)


class C(Foo, Bar):
    pass

The problem with using positional arguments is that mypy can't tell, if you were to make a call like C().run(1,2,3), what each call should do with 3. Think of C as a Foo, it should be the value of foo. But thinking of C as a Bar, it should be the value of bar. Generally, it's a bad idea to add positional arguments to the signature of an overriden method, even if at runtime 3 will be bound to the first available parameter of whichever method is called first.

Also, because there's no way to pass positional arguments for both foo and bar with an instance of C, you would have to pass them as keyword arguments anyway, so you may as well declare them as such.

Update

This may actually be a mypy bug. Nothing is statically typed in your code, so mypy should read the types of all arguments as Any and not report any errors.

In fact, if you use pyright, you will get no errors with this code: see the pyright playground code

NB: There are several different static type checkers for python:

  • mypy: The original (python core dev) static type checker
  • pyright: Microsoft's static type checker
  • pyre: Facebook's static type checker
  • pytype: Google's static type checker

and for enforcement of runtime type checking:


How are the two classes "incompatible"?

As @chepner correctly surmised, the issue is due to the conflict over positional versus keyword arguments, but his/her answer is not fully correct.

The reason they are incompatible is because they open the potential for a keyword collision in one method, but not the other. This is a kind of Liskov Violation.

Answer

Python distinguishes between parameters vs. arguments. Your function parameters are positional-or-keyword, meaning you can use them either by position or by keyword. (Sometimes authors have loosely used the term standard arguments when referring to arguments passed to positional-or-keyword parameters (like this realpython.com article), but that's not a term defined by python in their glossary.

I will use the mypy playground to support my claims.

If you remove **kwargs from the run functions, you will actually not get any errors:

This doesn't fail because, due to the MRO of cooperative multiple inheritance, C calls Foo first and Foo calls Bar because you use super. You can't even call c.run() with bar because it doesn't have it:

>>> c.run(1, 2, bar='baz')
Traceback (most recent call last):
  File "demo.py", line 27, in <module>
    c.run(1, 2, bar='baz')
TypeError: run() got an unexpected keyword argument 'bar'

This is all deterministic, so there's no potential for ambiguity here.

The problem occurs when you add **kwargs. Because of it, you can get one function to work, but not the other.

In every case, whenever you use **kwargs, you run the risk of accidentally passing a keyword argument that has the same name as an existing argument in your function. For example, if we bring back the **kwargs

...

class Foo(Parent):
    def run(self, a, b, foo, **kwargs):
        print('foo', foo)
        super().run(a, b, **kwargs)

...

and call c like so, we get an error:

>>> c.run(1, 2, 3, foo='baz')
Traceback (most recent call last):
  File "demo.py", line 27, in <module>
    c.run(1, 2, 3, foo='baz')
TypeError: run() got multiple values for argument 'foo'

Because your arguments are standard, foo can be either positional or keyword, so we confuse run() here with what we are doing.

If we consider the case of Bar with **kwargs added back:

class Bar(Parent):
    def run(self, a, b, bar, **kwargs):
        print('bar', bar)
        super().run(a, b, **kwargs)

We can call Bar.run() with a keyword argument of foo because foo is not in the parameter list of Bar.run():

>>> b = Bar()
>>> b.run(1, 2, 3, foo='baz')
bar 3

Hence, there is an incompatibility because you can call Bar one way, but not Foo with the same arguments (and vice-versa if we were to pass bar as a keyword argument).

There are two ways you can overcome this error. The first, shown by @chepner, will work if you want foo and bar to be keyword-only.

The second way is if you want to specify foo and bar to be positional only. Below are links to mypy playground with working examples:

Related