Rename column names using dictionary and regular expression

user_13860217 0
Viewed 158

I have a DataFrame called mcmc_samples which contains Markov-Chain-Monte-Carlo Samples for multiple variables

deviance K_1[1,1] K_1[1,2] K_1[1,3] K_1[2,1] K_1[2,2] K_1[2,3]
0.2 0.4 0.6 0.1 0.3 0.9 0.8
... ... ... ... ... ... ...

The columns names are composed of the level (K_1), the variable (first number in brackets before comma) and the category (second number in brackets after comma).

I try to rename the column names such that the numbers in the brackets are more meaningful. For this purpose, I want to use the following dictionaries.

dict_var = {1: "variable_1", 2: "variable_2"}
dict_categ  = {1: "item_1, 2: "item_2", 3: "item_3"}

I tried to replace the strings using regular expression

mcmc_samples.columns = mcmc_samples.columns.str.replace(r"(?<=,)(.*?)(?=\])", 
mcmc_samples.columns.str.extract(r"(?<=,)(.*?)(?=\])")[0].map(dict_categ), regex=True)

but this gave me the following error:

TypeError: repl must be a string or callable

pandas rename python regex string
1 Answers

Assuming the dictionaries contain comprehensive data and all numbers you will match will have corresponding keys in the dictionaries, you can use

mcmc_samples.columns = mcmc_samples.columns.str.replace(
    r"(?<=\[)(\d+),(\d+)(?=])",
    lambda x: f"{dict_var[int(x.group(1))]},{dict_categ[int(x.group(2))]}",
    regex=True)

See the regex demo. Details:

  • (?<=\[) - right before, there must be a [ char
  • (\d+) - Group 1: one or more digits
  • , - a comma
  • (\d+) - Group 2: one or more digits
  • (?=]) - right after, there must be a ] char.

If you need to return the matched number if it is not present in the dictionaries you can use

def repl(x):
    result = []
    if int(x.group(1)) in dict_var:
        result.append(dict_var[int(x.group(1))])
    else:
        result.append(x.group(1))
    if int(x.group(2)) in dict_categ:
        result.append(dict_categ[int(x.group(2))])
    else:
        result.append(x.group(2))
    return ",".join(result)

mcmc_samples.columns = mcmc_samples.columns.str.replace(
    r"(?<=\[)(\d+),(\d+)(?=])",
    lambda x: repl(x),
    regex=True)
user_38329700
Related