I have two columns with col1 and col2
I'm trying to create the key column to show that a and b is the same as b and a
Problem code:
import pandas as pd
pd.DataFrame({'Col1':['a','c','b','e'],
'Col2':['b','d','a','f']})
I have two columns with col1 and col2
I'm trying to create the key column to show that a and b is the same as b and a
Problem code:
import pandas as pd
pd.DataFrame({'Col1':['a','c','b','e'],
'Col2':['b','d','a','f']})
Try with frozenset
df['out'] = df.agg(frozenset,1).str.join(sep='')
Out[220]:
0 ab
1 dc
2 ab
3 ef
dtype: object
Use numpy to sort and then join/sum the strings. This creates a unique key regardless of the order the values appear across columns.
import numpy as np
df['key'] = np.sort(df.to_numpy(), axis=1).sum(1)
# Col1 Col2 key
#0 a b ab
#1 c d cd
#2 b a ab
#3 e f ef
In case you're interested numpy.sort is the performant way to do this. Any .agg or .apply reduce to slow loops over the rows so they will scale worse as the DataFrame becomes larger. Relative to numpy.sort the other methods are generally at least an order of magnitude slower.
import perfplot
import pandas as pd
import numpy as np
from string import ascii_lowercase
def apply_sorted(df):
return df.apply(sorted, 1).str.join(sep='')
def numpy_sort(df):
return pd.Series(np.sort(df.to_numpy(), axis=1).sum(1))
perfplot.show(
setup=lambda N: pd.DataFrame({'Col1': np.random.choice([*ascii_lowercase], N),
'Col2': np.random.choice([*ascii_lowercase], N)}),
kernels=[
lambda df: apply_sorted(df),
lambda df: numpy_sort(df),
],
labels=['apply_sorted', 'numpy_sort'],
n_range=[2 ** k for k in range(25)],
equality_check=lambda x,y: x.compare(y).empty,
relative_to=1,
xlabel='len(df)'
)
I would use sorted built-in functions for this task following way
import pandas as pd
df = pd.DataFrame({'Col1':['a','c','b','e'],'Col2':['b','d','a','f']})
df['key'] = df.apply(sorted,1).str.join(sep='')
print(df)
output
Col1 Col2 key
0 a b ab
1 c d cd
2 b a ab
3 e f ef
Explanation: it does sort every row lexicographically then join such elements. Assumption: every element of Col1 is single character and every element of Col2 is single character.