Type hint that methods in a subclass of requests.Session accept new parameters

user_1591921 0
Viewed 269

This code runs fine, but mypy complains:

class NamePrintingSession(requests.Session):
    def request(self, method, url, name=None, **kwargs):
        print(name)
        return super().request(method, url, **kwargs)


nps = NamePrintingSession()
nps.get("http://www.google.com", name="wooo") # error: Unexpected keyword argument "name" for "get" of "Session"

I kind of understand WHY it fails: Althought the definition of get() in requests accepts **kwargs, the stubs have some sort of additional information, and that doesnt change just because request() in my derived class accepts an additional parameter.

Is it possible to type-hint the get, post, delete, etc methods in my subclass with the parameters from the parent class but add the name parameter? (preferably without explicitly re-stating every parameter and also keeping the documentation links intact)

python-typing type-hinting python
2 Answers

The implementation of the get method at sessions.py is the following

# sessions.py

class Session:
    ...

    def get(self, url, **kwargs):
        r"""Sends a GET request. Returns :class:`Response` object.

        :param url: URL for the new :class:`Request` object.
        :param \*\*kwargs: Optional arguments that ``request`` takes.
        :rtype: requests.Response
        """

        kwargs.setdefault('allow_redirects', True)
        return self.request('GET', url, **kwargs)

So it uses the request method for its implementation, setting GET as a request method. Overriding the arguments of the request method changes only the typing behavior of the request method.

The base implementation allows any keyword argument as it defines **kwargs, but it also has an interface declared at the sessions.pyi file. .pyi files are called stub files and are used to separate implementation from typing interfaces. If the implementation does not contain any typing hints, the IDE looks at the .pyi declaration.

As we can see, all possible keyword arguments of the get method are declared at sessions.pyi.

# sessions.pyi

class Session:
    ...

    def get(
        self,
        url: Text | bytes,
        params: _Params | None = ...,
        data: Any | None = ...,
        headers: Any | None = ...,
        cookies: Any | None = ...,
        files: Any | None = ...,
        auth: Any | None = ...,
        timeout: Any | None = ...,
        allow_redirects: bool = ...,
        proxies: Any | None = ...,
        hooks: Any | None = ...,
        stream: Any | None = ...,
        verify: Any | None = ...,
        cert: Any | None = ...,
        json: Any | None = ...,
    ) -> Response: ...

The issue appeared as the interface did not contain the name keyword argument.

So what is happening when we call the get method on the object of the NamePrintingSession?

  • As the NamePrintingSession does not implement the get method, it will call the get method implementation of the Session class.
  • As the get implementation at sessions.py does not contain typing hints, the IDE looks for .pyi file of the sessions.py, which should be sessions.pyi, and if any interface of get is found, then uses it. If the get method did not have an interface, any keyword arguments would be allowed because of the **kwargs of the base implementation.

To fix that issue, you can customize the implementation of the get method for the NamePrintingSession class. But this solution is not optimal if the name argument should not affect the request behavior. In other words, if the name is used only in the request method, then there is no reason to override the rest methods like

# customSession.py

import requests


class NamePrintingSession(requests.Session):
    def request(self, method, url, name=None, **kwargs):
        print(name)
        return super().request(method, url, **kwargs)

    def get(self, url, name=None, **kwargs):
        print(name)
        super().get(url, **kwargs)

Another solution is that you can create a stub file and declare a new interface for your NamePrintingSession class. Suppose the file that contains the NamePrintingSession class is called customSession.py. Then you should create a file with the name customSession.pyi in the same directory.

# customSession.pyi

import requests


class NamePrintingSession(requests.Session):
    def request(self, method, url, name=None, **kwargs):
        ...

    def get(self, url, name=None, **kwargs):
        ...
user_127551870

Heavily inspired by Artyom's answer and combined with Unpack (only available in pyright atm, not yet in mypy), this is the best that can be done I think. But mostly I'll just use self.request("GET", ...) directly instead (because this solution still doesnt give proper code completion suggestions in vscode)

# mymodule.pyi

import requests
from typing import Any, Union
from typing_extensions import Unpack, TypedDict, NotRequired

class RequestArgs(TypedDict):
    name: NotRequired[str]
    params: NotRequired[Any]
    data: NotRequired[Any]
    headers: NotRequired[Any]
    cookies: NotRequired[Any]
    files: NotRequired[Any]
    auth: NotRequired[Any]
    timeout: NotRequired[Any]
    allow_redirects: NotRequired[bool]
    proxies: NotRequired[Any]
    hooks: NotRequired[Any]
    stream: NotRequired[Any]
    verify: NotRequired[bool]
    cert: NotRequired[Any]
    json: NotRequired[Any]

class NamePrintingSession(requests.Session):
    def get(self, url: Union[str, bytes], **kwargs: Unpack[RequestArgs]): ...
    def options(self, url: Union[str, bytes], **kwargs: Unpack[RequestArgs]): ...
    def head(self, url: Union[str, bytes], **kwargs: Unpack[RequestArgs]): ...
    def post(self, url: Union[str, bytes], **kwargs: Unpack[RequestArgs]): ...
    def put(self, url: Union[str, bytes], **kwargs: Unpack[RequestArgs]): ...
    def patch(self, url: Union[str, bytes], **kwargs: Unpack[RequestArgs]): ...
    def delete(self, url: Union[str, bytes], **kwargs: Unpack[RequestArgs]): ...
user_15919210
Related