How to efficiently generate this dictionary by partitioning a column?

user_7357673 0
Viewed 62

I have a dataframe df as follows

    parent_id   name
0   t3_35jfjt   t1_cr4y72v
1   t3_35jfjt   t1_cr4y7m7
2   t3_35jfjt   t1_cr4y7p3
3   t1_cr4y72v  t1_cr4y92z
4   t3_35jfjt   t1_cr4y986
... ...         ...

in which all elements in column name are unique. I would like to partition column name based on column parent_id. This means 2 elements in column name belong to the same class if they have the same parent_id.

Then I would like to create a dictionary whose keys are elements in column name and whose values are the number of elements of the class to which those elements belong. For example,

{'t1_cr4y72v': 3, 't1_cr4y72v': 4,...}

In this case, t1_cr4y72v belong to the class whose cardinality is 3.

I can use a loop to make this dictionary, but it's not efficient since I have over 3 millions rows. Could you please elaborate on an efficient method?

import pandas as pd
df = pd.read_csv('https://raw.githubusercontent.com/leanhdung1994/WebMining/main/df.csv', header = 0)
df
pandas dictionary python
2 Answers

Here is a solution:

dict(zip(df.name, df.groupby('parent_id')['parent_id'].transform('count')))
user_47215700

Another solution:

df.merge(
    df.groupby("parent_id")["name"].nunique().rename("cardinality"),
    on="parent_id"
).set_index("name")["cardinality"].to_dict()
user_110724440
Related