Rename all columns after all columns aggregation

user_8685939 0
Viewed 1011

I do an aggregation sum on all columns in a dataframe with a code similar like this:

   df_aggregated = df.groupBy('key').sum()

Result columns have changed his name for a pattern like this:

   sum(colum1), sum(colum2)

I need change all new column's name to previous name

I tried to use 'alias', but I have too many columns and I would like something more automatic

pyspark apache-spark dataframe aggregate python
1 Answers

You can use this function. Any aggregation have the same pattern: xxx(column_name) If you have a pattern use a regular expression to match it.

import re

def rename_columns_after_any_aggregation(df):
    for name in df.schema.names:
        clear_name = ''
        m = re.search('\((.*?)\)', name)
        if m:
            clear_name = m.group(1)
            df = df.withColumnRenamed(name, clear_name)

    return df
user_18276900
Related