Suppose I have a data frame, and 3 of its columns contain parameters. Thus all possible cohorts are combinations: suppose the columns have "duck", "cat", "dog", another "white", "grey", "black", the last one "small", "big". Thus, cohorts are small grey duck, big black cat, and so on.
Thus, how can group the data frame by all possible cohorts stored therein? I will need this to calculate certain values for all cohorts. Thank you.
Somehow you have to split your series into unique elements (whether those are lists or pandas.Series).
For example :
cols1 = set(df['col1'])
cols2 = set(df['col2'])
cols3 = set(df['col3'])
df = pd.DataFrame([[x,y,z] for x in cols1 for y in cols2 for z in cols3], columns=['col1', 'col2', 'col3'])
df.drop_duplicates(keep="first", inplace=True)
You could also do this using a loop (looping over df.columns.tolist()) and storing series values into a dict.
(This help you gather all the possible combinations of cohorts)
EDIT :
To group combinations in you first dataframe, you could do something like this :
df.groupby(['col1', 'col2', 'col3'])['col4'].mean()
See pandas.groupby's doc
EDIT 2 :
Another way to get all your combinations could be (using pandas only) :
from functools import reduce
cols = [df[[col]].drop_duplicates(keep='first') for col in ['col1', 'col2', 'col3']]
for sub_df_col in cols:
sub_df_col['CARTESIAN_PRODUCT'] = 1 #you now have a new column which the same value everywhere
df2 = reduce(lambda left,right: pd.merge(left,right,on=['CARTESIAN_PRODUCT'], how='outer'), cols).drop('CARTESIAN_PRODUCT', axis=1)
Credits to @everestial007 for the reduce/lambda solution