mypy - callable with derived classes gives error

Let's talk about type variance. Under typical subtyping rules, if we have a type DerivedClass that is a subtype of a type BaseClass, then every instance of DerivedClass is an instance of BaseClass. Simple enough, right? But now the complexity arises when we have generic type arguments.

Let's suppose that we have a class that gets a value and returns it. I don't know how it gets it; maybe it queries a database, maybe it reads the file system, maybe it just makes one up. But it gets a value.

class Getter:
    def get_value(self):
        # Some deep magic ...

Now let's assume that, when we construct the Getter, we know what type it should be querying at compile-time. We can use a type variable to annotate this.

T = TypeVar("T")
class Getter(Generic[T]):
    def get_value(self) -> T:
        ...

Now, Getter is a valid thing. We can have a Getter[int] which gets an integer and a Getter[str] which gets a string.

But here's a question. If I have a Getter[int], is that a valid Getter[object]? Surely, if I can get a value as an int, it's easy enough to upcast it, right?

my_getter_int: Getter[int] = ...
my_getter_obj: Getter[object] = my_getter_int

But Python won't allow this. See, Getter was declared to be invariant in its type argument. That's a fancy way of saying that, even though int is a subtype of object, Getter[int] and Getter[object] have no relationship.

But, like I said, surely they should have a relationship, right? Well, yes. If your type is only used in positive position (glossing over some details, that means roughly that it only appears as the return value of methods or as the type of read-only properties), then we can make it covariant.

T_co = TypeVar("T_co", covariant=True)
class Getter(Generic[T_co]):
    def get_value(self) -> T_co:
        ...

By convention, in Python, we denote covariant type arguments using names that end in _co. But the thing that actually makes it covariant here is the covariant=True keyword argument.

Now, with this version of Getter, Getter[int] is actually a subtype of Getter[object]. In general, if A is a subtype of B, then Getter[A] is a subtype of Getter[B]. Covariance preserves subtyping.

Okay, that's covariance. Now consider the opposite. Let's say we have a setter which sets some value in a database.

class Setter:
    def set_value(self, value):
        ...

Same assumptions as before. Suppose we know what the type is in advance. Nowe we write

T = TypeVar("T")
class Setter:
    def set_value(self, value: T) -> None:
        ...

Okay, great. Now, if I have a value my_setter : Setter[int], is that a Setter[object]? Well, my_setter can always take an integer value, whereas a Setter[object] is guaranteed to be able to take any object. my_setter can't guarantee that, so it's actually not. If we try to make T covariant in this example, we'll get

error: Cannot use a covariant type variable as a parameter

Because it's actually not a valid relationship. In fact, in this case, we get the opposite relationship. If we have a my_setter : Setter[object], then that's a guarantee that we can pass it any object at all, so certainly we can pass it an integer, hence we have a Setter[int]. This is called contravariance.

T_contra = TypeVar("T_contra", contravariant=True)
class Setter:
    def set_value(self, value: T_contra) -> None:
        ...

We can make our type contravariant if it only appears in negative position, which (again, oversimplifying a bit) generally means that it appears as arguments to functions, but not as a return value. Now, Setter[object] is a subtype of Setter[int]. It's backwards. In general, if A is a subtype of B, then Setter[B] is a subtype of Setter[A]. Contravariance reverses the subtyping relationship.

Now, back to your example. You have a Callable[[DerivedClass], str] and want to know if it's a valid Callable[[BaseClass], str]

Applying our principles from before, we have a type Callable[[T], S] (I'm assuming only one argument for simplicity's sake, but in reality this works in Python for any number of arguments) and want to ask if T and S are covariant, contravariant, or invariant.

Well, what is a Callable? It's a function. It has one thing we can do: call it with a T and get an S. So it's pretty clear that T is only used as an argument and S as a result. Things only used as arguments are contravariant, and those used as results are covariant, so in reality it's more correct to write

Callable[[T_contra], S_co]

Arguments to Callable are contravariant, which means that if DerivedClass is a subtype of BaseClass, then Callable[[BaseClass], str] is a subtype of Callable[[DerivedClass], str], the opposite relationship to the one you suggested. You need a function that can accept any BaseClass. A function with a BaseClass argument would suffice, and so would a function with an object argument, or any type which is a supertype of BaseClass, but subtypes are insufficient because they're too specific for your contract.

MyPy objects to your call of p with r as its argument because given only the type signatures, it can't be sure the function won't be called with a non-DerivedClass instance.

For instance, given the same type annotations, p could be implemented like this:

def p(q: Callable[[BaseClass], str]) -> None:
    obj = BaseClass()
    q(obj)

This will break p(r) if r has an implementation that depends on the derived attributes of its argument:

def r(derived: DerivedClass) -> str:
    return str(derived.q)