Can I use abstract methods to import file-specific formatting of (Python) pandas data?

user_2780360 0
Viewed 166

I have a class FileSet with a method _process_series, which contains a bunch of if-elif blocks doing filetag-specific formatting of different pandas.Series:

    elif filetag == "EntityA":
        ps[filetag+"_Id"] = str(ps[filetag+"_Id"]).strip()
        ps[filetag+"_DateOfBirth"] = str(pd.to_datetime(ps[filetag+"_DateOfBirth"]).strftime('%Y-%m-%d')).strip()
        ps[filetag+"_FirstName"] = str(ps[filetag+"_FirstName"]).strip().capitalize()
        ps[filetag+"_LastName"] = str(ps[filetag+"_LastName"]).strip().capitalize()
        ps[filetag+"_Age"] = relativedelta(datetime.today(), datetime.strptime(ps[filetag+"_DateOfBirth"], "%Y-%m-%d")).years
        return ps

I'd like to define an abstract format method in the class and keep these blocks of formatting in separate modules that are imported when _process_series is called for a given filetag. Forgive the pseudo-code, but something like:

for tag in filetag:
    from my_formatters import tag+'_formatter' as fmt
    ps = self.format(pandas_series, fmt)
    return ps

And the module would contain the formatting block:

# my_formatters.EntityA_formatter
    ps[filetag+"_Id"] = str(ps[filetag+"_Id"]).strip()
    ps[filetag+"_DateOfBirth"] = str(pd.to_datetime(ps[filetag+"_DateOfBirth"]).strftime('%Y-%m-%d')).strip()
    ps[filetag+"_FirstName"] = str(ps[filetag+"_FirstName"]).strip().capitalize()
    ps[filetag+"_LastName"] = str(ps[filetag+"_LastName"]).strip().capitalize()
    ps[filetag+"_Age"] = relativedelta(datetime.today(), datetime.strptime(ps[filetag+"_DateOfBirth"], "%Y-%m-%d")).years
    return ps
pandas python-module methods format python
3 Answers

You can create a function in it's own .py file and import it. If you create the same function in each file you can then call it.

here is f1.py:

def gimme():
    return 'format 1'

here is f2.py:

def gimme():
    return 'format 2'

Then you main file:

module_names = ['f1','f2']

for module_name in module_names:
    import_test = __import__(module_name)
    result = import_test.gimme()
    result = import_test.gimme()
    print(result)

Which gives the output:

format 1
format 2
user_145600400

Why not use globals with asterisk:

from my_formatters import *

for tag in filetag:
    fmt = globals()[tag + '_formatter']
    ps = self.format(pandas_series, fmt)
    return ps

I converted your pseudocode to real code.

globals documentation:

Return a dictionary representing the current global symbol table. This is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).

user_87083640

Your psuedocode could be made into real code like so:

import my_formatters

for tag in filetag:
    fmt = getattr(my_formatters, tag + '_formatter')
    ps = self.format(pandas_series, fmt)
    return ps
user_59469210
Related