Pythonic way of composing function calls with multiple arguments inside classes

Question

According to this video (https://youtu.be/ka70COItN40?t=1185), it is not a good idea to store many intermediate variables with the same name in a program because the state of this variable would change during the execution of the program.

A better way, according to the author of the video, would be to use reduce from functools and define compose() function to chain the different data transformations.

To illustrate with an example, imagine that I'm writing a program to scrape some text from the web, and I need to clean it for analyzing it later. The "wrong" way to do it would be to implement it along the following lines:

import regex
from html import unescape
from bs4 import BeautifulSoup

class MyAssignedClass():
    def __init__(self, description):
        self.description = description

    def clean_description(self):
        self.description = unescape(self.description)
        self.description = BeautifulSoup(self.description, "html.parser").text
        self.description = regex.sub("
", " ", self.description)
        self.description = regex.sub("\xa0", ' ', self.description)
        self.description = regex.sub(" +", " ", self.description)

def main():
    text = "I'm full of      
 \xa0 things to clean & to skip."
    
    A = MyAssignedClass(description=text)
    A.clean_description()
    print(f'MyAssignedClass.description: {A.description}')

if __name__ == "__main__":
    main()

Output:

>>> MyAssignedClass.description: I'm full of things to clean & to skip.

If I follow the guidelines proposed in the video, I should create a compose() function instead to chain the transformations. The problem here is that I must use lambda functions everywhere to make it work:

import regex
from html import unescape
from bs4 import BeautifulSoup
from typing import Callable
import functools

ComposableFunction = Callable[[str], str]

def compose(*functions: ComposableFunction) -> ComposableFunction:
    return functools.reduce(lambda f, g: lambda x: g(f(x)), functions)

class MyComposedClass():
    def __init__(self, description):
        self.description = description

    def clean_description(self):
        myfunc = compose(
            unescape, 
            lambda _: BeautifulSoup(_, features="html.parser").text,
            lambda _: regex.sub("
", " ", _),
            lambda _: regex.sub("\xa0", ' ', _),
            lambda _: regex.sub(" +", " ", _)
            )

        self.description = myfunc(self.description)

def main():
    text = "I'm full of      
 \xa0 things to clean & to skip."
    
    B = MyComposedClass(description=text)
    B.clean_description()
    print(f'MyComposedClass.description: {B.description}')

if __name__ == "__main__":
    main()

Output:

>>> MyComposedClass.description: I'm full of things to clean & to skip.

However, wouldn't it be more readable to just use intermediate variables and assign self.description only at the end of the method?

import regex
from html import unescape
from bs4 import BeautifulSoup

class MyUnderAssignedClass():
    def __init__(self, description):
        self.description = description

    def clean_description(self):
        _ = unescape(self.description)
        _ = BeautifulSoup(_, "html.parser").text
        _ = regex.sub("
", " ", _)
        _ = regex.sub("\xa0", ' ', _)
        _ = regex.sub(" +", " ", _)
        self.description = _

def main():
    text = "I'm full of      
 \xa0 things to clean & to skip."
    
    C = MyUnderAssignedClass(description=text)
    C.clean_description()
    print(f'MyUnderAssignedClass.description: {C.description}')

if __name__ == "__main__":
    main()

Output:

>>> MyUnderAssignedClass.description: I'm full of things to clean & to skip.

Which are the most pythonic way of writing such piece of code? What are the pitfalls associated with using intermediate variables (and reasssigning the attribute many times within a method)?

user_8219760 · Answer

I disagree with the video author. This question is a great one but the suggestion from the video that x = func(x) should not be used is false in my opinion. For example in numpy it is completely fine to do these transformation with x = func(x) style. In the video example the Compose allows simplification of forward method and in a case where the network would be more complex (hundreds of layers) the original approach would be unpractical.

Let's see an example

class A:
    def __init__(self):
        # if these were all saved to class with `self.a = func1` or equivalent 
        # this would be unmaintainable
        self.functions = [generate_random_transform_function(x) for x in range(int(1e3))]
        # This combines the functions to be called all at once
        self.transform = compose(self.functions)
    def forward(self, x):
        # with list func
        # for f in self.functions:
        #    x = f(x)
        # return x
        # Or more horrible
        # x = self.x1(x)
        # x = self.x2(x)
        # 

        # compared to
        return self.transform(x)

This question is partly about functional versus object oriented approach to coding both of which are fine in python. Personally I prefer the more functional x = f(x) style.

Pythonic way of composing function calls with multiple arguments inside classes

1 Answers