Using signature of implemented method on Derived class for inherited method from Base class in a "template method pattern" inheritance in Python

user_3697191 0
Viewed 119

Consider we have Template method pattern inheritance scenario. I am using an example with requests library.

import abc
import json
import requests
from typing import Dict, Any

class Base(abc.ABC)
    @abc.abstractmethod
    def call(self, *args, **kwargs) -> requests.Response:
        raise NotImplementedError

    def safe_call(self, *args, **kwargs) -> Dict[str, Any]:
        try:
            response = self.call(*args, **kwargs)
            response.raise_for_status()
            return response.json()
        except json.JSONDecodeError as je: ...
        except requests.exceptions.ConnectionError as ce: ...
        except requests.exceptions.Timeout as te: ...
        except requests.exceptions.HTTPError as he: ...
        except Exception as e: ...
        return {"success": False}


class Derived(Base):
    def call(self, url: str, json: Dict[str, Any], timeout: int, retries: int) -> requests.Response:
        # actual logic for making the http call
        response = requests.post(url, json=json, timeout=timeout, ...)
        return response

which will be used like

executor = Derived()
response = executor.safe_call(url='https://example.com', json={"key": "value"}, ...)

Is there a way to attach the signature of Derived.call to Derived.safe_call so modern IDE auto-complete and type checkers work? Because this awfully looks like a decorator pattern, I tried using functools.update_wrapper in __init__ but that fails with

AttributeError: 'method' object has no attribute '__module__'

I do have a few options in mind, but I am unable to find any recommendations on this problem

  1. I can go the metaclass route to change up signature and docstring attributes before the class is completely constructed. However, this might not work well with IDEs.
  2. I can define a stub file .pyi and maintain it along with the main code.
class Derived(Base):
    def safe_call(self, url: str, json: Dict[str, Any], timeout: int, retries: int) -> Dict[str, Any]: ...

  1. Change the design to something else completely

  2. (Edit: Added post first answer) (Ab)use @typing.overload

class Derived(Base):
    @typing.overload
    def safe_call(self, url: str, json: Dict[str, Any], timeout: int, retries: int) -> Dict[str, Any]:
        ...

    def call(self, url: str, json: Dict[str, Any], timeout: int, retries: int) -> requests.Response:
        # actual logic for making the http call
        response = requests.post(url, json=json, timeout=timeout, ...)
        return response
python-typing python
1 Answers

I can see what you're trying to do, but I think it is probably a mistake to go down that route, as it would break the Liskov Substitution Principle.

Currently, your abstract class Base defines an interface in which the method Base.call can, in concrete implementations of the interface, be called with any positional or keyword arguments without raising an error at runtime. From a theoretical point of view, your concrete implementation of call in your Derived class does not satisfy this interface. If it is called with >4 arguments, called with <4 arguments, or called with keyword-arguments that are not url, json, timeout or retries, it will raise an error at runtime.

Python will nonetheless let you instantiate instances of Derived at runtime, as it only checks for the presence of a method by the same name as the abstract method defined in Base. It does not check for signature compatibility of abstract methods upon instantiation of a concrete implementation, and attempting to do so would be extremely difficult. However, some type checkers may complain that the signature in your derived class is less permissive than in your base class. Moreover, the theoretical principle is important in its own right.

I think the way to go would probably be something like this. Note two things here:

  1. The safe_call and call methods in the base class have both become private methods, as they are implementation details that are not designed to be used directly.
  2. I have altered the signature of call and safe_call in the base class so that they only accept keyword arguments.
import abc
import json
import requests
from typing import Dict, Any

class Base(abc.ABC)
    @abc.abstractmethod
    def _call(self, **kwargs: Any) -> requests.Response:
        raise NotImplementedError

    def _safe_call(self, **kwargs: Any) -> Dict[str, Any]:
        try:
            response = self.call(**kwargs)
            response.raise_for_status()
            return response.json()
        except json.JSONDecodeError as je: ...
        except requests.exceptions.ConnectionError as ce: ...
        except requests.exceptions.Timeout as te: ...
        except requests.exceptions.HTTPError as he: ...
        except Exception as e: ...
        return {"success": False}


class Derived(Base):
    def _call(self, **kwargs: Any) -> requests.Response:
        url: str = kwargs['url']
        json: Dict[str, Any] = kwargs['json']
        timeout: int = kwargs['timeout']
        retries: int = kwargs['retries']
        
        # actual logic for making the http call
        response = requests.post(url, json=json, timeout=timeout, ...)
        return response

    def safe_call_specific_to_derived(self, url: str, json: Dict[str, Any], timeout: int, retries: int) -> Dict[str, Any]:
        return self._safe_call(url=url, json=json, timeout=timeout, retries=retries)

If you want to have default values for parameters in your concrete implementations of _call, note that you can use kwargs.get(<arg_name>, <default_val>) instead of kwargs[<arg_name>]

user_139900160
Related