Python Concept for a big Workflow/Pipeline

Viewed 33

I don't want to reinvent the wheel and code too chaotic.

I want to automate following Workflow:

  1. get File from directory, transform & split in multiple files
  2. upload on a web app and download new generated files (using selenium, because no official api and too much session cookies)
  3. transform downloaded files and merge them
  4. upload on a sharepoint
  5. AND I want to log every step

I wrote all my functions and everything is working fine but I want to code clean by using any programming concept (eg. OOP). I tried something like below but I'm not happy with it. Can some one give me some source/advice how to clean code my problem?

example of my concept:

imports ...

CONSTANT_VARIABLES ...

load_dotenv() ...

Chromedriver setup for selenium


class Workflow:
    def logging_function():
        ...

    def __init__():
        run_function1()
        run_function2()
        run_subflow()
        ...


    def run_function1():
        log = logging_function()
        ...
    def run_function2():
        log = logging_function()
        ...
    def run_subflow():
        run_subflow_function1()
        run_subflow_function2()
        ...

def start_workflow():
    Workflow()

if __name__ == "__main__":
    start_workflow()
0 Answers
Related