Object-Oriented Design For Data Frame Transformation

user_15935501 0
Viewed 289

I'm trying to apply object oriented design to my data science project. For data cleaning, we usually start off with a table(dataframe) and then do transformations on this dataframe(For example, creating a new column based on existing column, add two columns together etc).

My initial thought on this (in python) was basically create a class Data that would have an underlying pandas dataframe as a private member and provide extra methods in the class to transform the data. After i reached this step, i was wondering:

  1. Am i doing object-oriented programming for the sake of doing it? Does this bring any advantage in comparison to just writing the code in pure pandas and forget about creating my own class?
class Data:

    def __init__(self, df):

        self._df = df

    def clean_dataframe(self):
        """
            Does some operation on self._df, eg adds a new column
            and sets self._df to the updated dataframe
        """
        self._df = some_operations(self._df)

Or just do:

def clean_dataframe(df):
    df = some_operations(df)
    return df

df = clean_dataframe(df)
  1. On a completely different note. Once we do a transformation from dataframe(d0) to dataframe(d1), in a object oriented sense, do i need to create a new class for d1 since the underlying dataframe has changed(for example, added a new column). In the same example , note that we redefined self._df after the clean_dataframe method.
    Should I be changing this variable or should be creating a new variable or even a new class to represent the transformed data.

Very new to Object-Oriented design and would love to gain more perspective.

pandas oop python
1 Answers

Compiling several helper functions for data processing, definitely makes sense, especially if there is need for repetitive use. I am not quite sure whether creating a Data class offers any exquisite advantages, unless there are other objects one would like to "bundle" ala C-type struct.

That said, for modelling purposes one might often resort to writing custom made estimators, when off-the-shelf doesn't seem adequate.

With respect to 2, in my opinion you don't need to create a separate dataframe, especially if the only difference is an extra column. As a cautionary tangent, one issue I have occasionally faced however is dataframes becoming too clunky, which might even affect IDE performance. But this also depends on the datatypes and size of the original dataframe.

user_81886660
Related