An appropriate way to let users of a module to override certain functions in Python

WZH 0
Viewed 47

I'm developing a Python module which would be pip installed into end user's environment. For part of my module, I would like to let the user override with some functions so that the system would be tailored to specific end-user's infrastructure.

For example, we might want users to specific some rule to determine the name of an s3 bucket. Basically somehow, let user define a function like the following:

def determine_s3(var1: str, var2: str) -> str:
    if var1 == "banana":
        return "fruits_s3_bucket"

    if var2 == "dog":
        return "animals_s3_bucket"

    return "others_bucket"

So different users of my module might have different aforementioned rules. They would have the same input arguments though (i.e. in this example, var1 and var2).

I also don't want to pass var1 and var2 everywhere when I need to use s3 because this is kinda static.

How can I setup my module to implement this in Python appropriately? Thanks.

P.S. In Java, I would probably let the user pass in a class name like "com.example.SomeClass" as a config and as long as the corresponding jar is on the classpath, it should work, but I'm not sure how to do that in Python.

python
3 Answers

Allow users to pass a custom function of their own and return the result of that with the arguments given to determine_s3.

from typing import Callable

def determine_s3(var1: str, var2: str, user_func: Callable[[str, str], str]) -> str:
    return user_func(var1, var2)

def sample_func(var1: str, var2: str) -> str:
    if var1 == "banana":
        return "fruits_s3_bucket"

    if var2 == "dog":
        return "animals_s3_bucket"

    return "others_bucket"

print(determine_s3("banana", "apple", sample_func))

Output:

fruits_s3_bucket
JRose0

If I understand you correctly, you should create an abstract class with the abstract method determine_s3:

from abc import ABC, abstractmethod
 
class BucketSelector(ABC):
    @abstractmethod
    def determine_s3(self, var1: str, var2: str) -> str:
        pass

And you'd use it in your library:

def do_something(bucket_selector: BucketSelector):
    bucket = bucket_selector.determine_s3(var1, var2)

The user would need to inherit that class with his implementation of determine_s3:

from your_beautiful_module import do_something, BucketSelector


class MyBucketSelector(BucketSelector):
    def __init__(self, banana):
        self.banana = banana

    def determine_s3(self, var1: str, var2: str) -> str:
        if var1 == self.banana:
            return "fruits_s3_bucket"
    
        if var2 == "dog":
            return "animals_s3_bucket"
    
        return "others_bucket"

do_something(bucket_selector=MyBucketSelector('banana'))

This would be a very explicit and versatile solution.

Yevhen Kuzmovych0

Use a config file is typically what people would do.

Yes, I know it doesn't exactly follow your var1 and var2 example, but the use of 2 vars to point to 1 s3 bucket is somewhat unclear to me, though the mechanism could be extended to cover multiple vars, if the logic remained the same for each user.

(I used json here as it is builtin, but yaml or toml might be better fits for end user configuration).

test.py
from pathlib import Path
import json
import sys

if len(sys.argv) >=2:
    var1 = sys.argv[1] 
else:
    var1 = ""

def determine_s3(var1: str, config: Path|None =None) -> str:

    pa = Path(config) if config else Path('~/config.json').expanduser()
    with pa.open() as fi:
        config = json.load(fi)

    res = config.get("buckets",{}).get(var1) or config["default_bucket"]
    print(f"determine_s3({var1=})=>{res}")

determine_s3(var1)
~/config.json:
{
    "buckets" : { 
        "banana": "fruits_s3_bucket",
        "dog": "animals_s3_bucket"
        
    },
    "default_bucket" : "others_bucket"
}

ouput:

% py test.py banana
determine_s3(var1='banana')=>fruits_s3_bucket
% py test.py dog   
determine_s3(var1='dog')=>animals_s3_bucket
% py test.py    
determine_s3(var1='')=>others_bucket
JL Peyret0
Related