Re-write pandas.DataFrame based code on numpy

userU1lC 0
Viewed 36

I'm writing custom metric function and here's the steps I implemented:

  1. I have a list of floats in preds and list of int 0-1 values in target
  2. I round preds
  3. I need to make groupby on preds
  4. Count mean target values for those groupedby preds
  5. Count MSE between groupedby preds and target

That's how df looks like before groupby

enter image description here

rounded = [np.round(x, 2) for x in preds]

df = pd.DataFrame({'target': target, 'preds': rounded})
        
df = df.groupby('preds')['target'].mean().to_frame().reset_index()
        
mse = mean_squared_error(df['target'], df['preds'])

And that's how after groupby and mean() (as I can't properly display groupby)

enter image description here

Basicaly, I don't know how to groupby on two python lists.

I did groupby on one list like that

gr_list = [list(j) for i, j in groupby(rounded)]

But I have no clue how to groupby second list, based on gr_list groupping

group-by list dataframe pandas python
1 Answers

Not the cleanest code, but I managed to do it like that:

from collections import defaultdict

d = defaultdict(list)
for i, item in enumerate(rounded): # rounded is rounded preds
    d[item].append(target[i])

enter image description here

meanDict = {}
for k,v in d.items():
    meanDict[k] = sum(v)/ float(len(v))

enter image description here

preds, target = zip(*avgDict.items())

mse = mean_squared_error(values, keys)
userU1lC0
Related