Python type hints: What does Callable followed by a TypeVar mean?

user_10931692 0
Viewed 5465

I am trying to understand the type hint Getter[T] in the following piece of code:

Simplified example

T = TypeVar('T')
Getter = Callable[[T, str], str]


class AbstractClass(abc.ABC):
    @abc.abstractmethod
    def extract(
        self,
        get_from_carrier: Getter[T],  #  <---- See here
        ...
    ) -> Context:

Help much appreciated since I have been breaking my head over this.

Original source code

The original source code is from the OpenTelemetry project file "textmap.py":

import abc
import typing

from opentelemetry.context.context import Context

TextMapPropagatorT = typing.TypeVar("TextMapPropagatorT")

Setter = typing.Callable[[TextMapPropagatorT, str, str], None]
Getter = typing.Callable[[TextMapPropagatorT, str], typing.List[str]]


class TextMapPropagator(abc.ABC):
    """This class provides an interface that enables extracting and injecting
    context into headers of HTTP requests. 
    ...
    """

    @abc.abstractmethod
    def extract(
        self,
        get_from_carrier: Getter[TextMapPropagatorT],
        carrier: TextMapPropagatorT,
        context: typing.Optional[Context] = None,
    ) -> Context:
python-typing type-hinting python generics
2 Answers

A Callable followed by a type variable means that the callable is a generic function that takes one or more arguments of generic type T.

The type variable T is a parameter for any generic type.

The line:

Getter = Callable[[T, str], str]

defines Getter as a type alias for a callable function whose arguments are of generic type T and string, and whose return type is string.

Therefore, the line:

get_from_carrier: Getter[T]

defines an argument (get_from_carrier) that is a generic function. And the first argument of the generic function is of generic type T.

Concrete Example

This can be better understood by looking at a concrete example. See propagators.extract below from "instrumentation/opentelemetry-instrumentation-asgi/src/opentelemetry/instrumentation/asgi/init.py ":

In the call propagators.extract, the function get_header_from_scope is a callable function whose first argument is of type dict, and this dict is serving as a TextMapPropagatorT.

def get_header_from_scope(scope: dict, header_name: str) -> typing.List[str]:
    """Retrieve a HTTP header value from the ASGI scope.

    Returns:
        A list with a single string with the header value if it exists, else an empty list.
    """
    headers = scope.get("headers")
    return [
        value.decode("utf8")
        for (key, value) in headers
        if key.decode("utf8") == header_name
    ]


...


class OpenTelemetryMiddleware:
    """The ASGI application middleware.
    ...
    """

    ...

    async def __call__(self, scope, receive, send):
        """The ASGI application  ... """
        if scope["type"] not in ("http", "websocket"):
            return await self.app(scope, receive, send)

        token = context.attach(
            propagators.extract(get_header_from_scope, scope)
        )
user_11644650

tl;dr: _C[_T] is a generic type alias that's equivalent to Callable[[_T, int], int].

Here you're defining _C to be the type alias of Callable[[_T, int], int]. When an type alias contains a TypeVar (in this case, _T), it becomes a generic type alias. You can use it the same way as you'd use built-in generic types like List[T] or Dict[K, V], for example, _C[str] would be equivalent to Callable[[str, int], int].

Then, type annotations of get_from_elem define it as a generic function. What this means is the the same type variable used within the entire function should be bound to the same class. To explain what this means, take a look at these function calls:

_T = typing.TypeVar('_T')
_C = typing.Callable[[_T,int],int]

def get_from_elem(get: _C[_T], elem: _T):
    ...

def foo_str(a: str, b: int) -> int:
    # This function matches `_C[str]`, i.e. `Callable[[str, int], int]`
    ...

def foo_float(a: float, b: int) -> int:
    # This function matches `_C[float]`, i.e. `Callable[[float, int], int]`
    ...

def foo_generic(a: _T, b: int) -> int:
    # This function matches `_C[_T]`, it is also a generic function
    ...

_T2 = typing.TypeVar('_T2', str, bytes)

def foo_str_like(a: _T2, b: int) -> int:
    # A generic function with constraints: type of first argument must be `str` or `bytes`
    ...


get_from_elem(foo_str, "abc")      # Correct: `_T` is bound to `str`
get_from_elem(foo_float, 1.23)     # Correct: `_T` is bound to `float`

get_from_elem(foo_str, 1.23)       # Wrong: `_T` bound to two different types `str` and `float`
get_from_elem(foo_float, [1.23])   # Wrong: `_T` bound to two different types `float` and `List[float]`

get_from_elem(foo_generic, 1.45)   # Correct: `_T` is only bound to `float`
get_from_elem(foo_str_like, 1.45)  # Wrong: `_T` is only bound to `float`, but doesn't satisfy `foo_str_like` constraints

In the last two cases, the first argument is a generic function, which does not bind the type variable, so the type variable is only bound by the second argument. However, in the last case, foo_str_like has an additional constraint on its first argument type, and the bound type float does not satisfy that constraint, so it fails type checking.

user_49092280
Related