Simply put, what are the preferred practices for writing larger python applications that use pandas dataframes as its primary method for data representation?
I often find myself struggling to maintain inconsistencies in dataframes, sometimes invariants leak through in data, datatypes are not what you expect etc.
I'm wondering just what are the best practices for writing larger, stable applications in pandas? I want to take advantage of array-representation in data for speed, but I also want to make sure that there's a way to further define the "bounds" of dataframe, what it should have in it, in a clean way.
- Assertions on receiving a dataframe from a caller.
- Forcing a dataframe parameter to have specific dtypes.
- Defining a dataframe "type" based upon the columns it has.
- Opportunities for OOP, at the dataframe level
Also, sorry for the vague nature of this. I'm starting on a project, and I want to ask this question before I get too far off course. I've been burned in the past with regards to not enforcing enough of a structure when it comes to dataframes.