Creating a unique value per row in pandas?

Viewed 3300

Acquire raw data --> transform it and join it with other files --> email to end-users for review

What is the best approach?

3 Answers

If anyone is looking for a modularized function, save this into a file for use where needed. (for Pandas DataFrames)

df is your dataframe, columns is a list of columns to hash over, and name is the name of your new column with hash values.

Returns a copy of the original dataframe with a new column containing the hash of each row.

def hash_cols(df, columns, name="hash"):
    new_df = df.copy()
    def func(row, cols):
        col_data = []
        for col in cols:
            col_data.append(str(row.at[col]))

        col_combined = ''.join(col_data).encode()
        hashed_col = sha256(col_combined).hexdigest()
        return hashed_col

    new_df[name] = new_df.apply(lambda row: func(row,columns), axis=1)

    return new_df

I had a similar problem, that I solved thusly:

import hashlib
import pandas as pd
df = pd.DataFrame.from_dict({'mine': ['yours', 'amazing', 'pajamas'], 'have': ['something', 'nothing', 'between'], 'num': [1, 2, 3]})
hashes = []
for index, row in df.iterrows():
    hashes.append(hashlib.md5(str(row).encode('utf-8')).hexdigest())
# if you want the hashes in the df, 
# in my case, I needed them to form a JSON entry per row
df['hash'] = hashes

The results will form an md5 hash, but you can really use any hash function you need to.

Related