Evaluating an Expression using data frames

We can try creating an indicator DataFrame using a Series.get_dummies to create indicator columns for each value in the ID column, then reduce to a single row per Patient via groupby max:

# Convert to ID columns to binary indicators
indicator_df = df.set_index('Patient')['ID'].str.get_dummies()
# Reduce to 1 row per Patient
indicator_df = indicator_df.groupby(level=0).max()

indicator_df:

         71  72  74  SD75
Patient                  
A         0   1   1     1
B         1   0   0     0
C         0   1   0     0

Now we can reindex from the expression terms to create missing columns. np.unique is used to ensure that duplicate terms in the expression do not result in duplicate columns in indicator_df (this can be omitted if it is guaranteed there are no duplicate terms):

exp = '(((71+72)*((73+75)+SD75))*((74+76)+SD76))'
# Extract terms from expression
cols = re.sub(r'[^\w]', ' ', exp).split()
# Convert to ID columns to binary indicators
indicator_df = df.set_index('Patient')['ID'].str.get_dummies()
# Reduce to 1 row per Patient
indicator_df = indicator_df.groupby(level=0).max()
# Ensure All expression terms are present
indicator_df = indicator_df.reindex(
    columns=np.unique(cols),  # prevent duplicate cols
    fill_value=0  # Added cols are filled with 0
)

indicator_df:

         71  72  73  74  75  76  SD75  SD76
Patient                                    
A         0   1   0   1   0   0     1     0
B         1   0   0   0   0   0     0     0
C         0   1   0   0   0   0     0     0

Now if we alter the exp slightly by surrounding these new columns names with backticks (`) we can use DataFrame.eval to compute the expression:

exp = '(((71+72)*((73+75)+SD75))*((74+76)+SD76))'
# Extract terms from expression
cols = re.sub(r'[^\w]', ' ', exp).split()
# create indicator_df (chained)
indicator_df = (
    df.set_index('Patient')['ID']
        .str.get_dummies()
        .groupby(level=0).max()
        .reindex(columns=np.unique(cols), fill_value=0)
)
# Eval the expression and create the resulting DataFrame
result = indicator_df.eval(
    # Add Backticks around columns names
    re.sub(r'(\w+)', r'`\1`', exp)
).reset_index(name='FinalVal')

result:

  Patient  FinalVal
0       A         1
1       B         0
2       C         0

The backticks are necessary to indicate these values represent column names, and not numeric values:

re.sub(r'(\w+)', r'`\1`', exp)

# (((`71`+`72`)*((`73`+`75`)+`SD75`))*((`74`+`76`)+`SD76`))

Notice the difference between 71 with backticks vs without:

# Column '71' + the number 71
pd.DataFrame({'71': [1, 2, 3]}).eval('B = `71` + 71')

   71   B
0   1  72
1   2  73
2   3  74

Alternatively, the indicator_df can be created with a crosstab and clip:

exp = '(((71+72)*((73+75)+SD75))*((74+76)+SD76))'
# Extract terms from expression
cols = re.sub(r'[^\w]', ' ', exp).split()
indicator_df = (
    pd.crosstab(df['Patient'], df['ID'])
        .clip(upper=1)  # Restrict upperbound to 1
        .reindex(columns=np.unique(cols), fill_value=0)
)
# Eval the expression and create the resulting DataFrame
result = indicator_df.eval(
    # Add Backticks around columns names
    re.sub(r'(\w+)', r'`\1`', exp)
).reset_index(name='FinalVal')

Setup and imports used:

import re

import numpy as np
import pandas as pd

df = pd.DataFrame({
    'Patient': ['A', 'A', 'A', 'A', 'B', 'C'],
    'ID': ['72', 'SD75', '74', '74', '71', '72']
})

I would not try to parse that expression and evaluate it. Instead, I would create dummy or indicator variables for the ID column. (Indicator variables are also called one-hot encoded variables.) With these indicators, you can then calculate your expression using a standard function.

Here's how to do it with Pandas and scikit-learn. I am using scikit-learn's OneHotEncoder. An alternative might be Panda's get_dummies(), but the OneHotEncoder allows you to specify the categories.

import pandas as pd
from sklearn.preprocessing import OneHotEncoder

variables = [71, 72, 73, 74, 75, 76, "SD75", "SD76"]
enc = OneHotEncoder(categories=[variables], sparse=False)
df = pd.DataFrame({
    "Patient": ["A"] * 4 + ["B", "C"],
    "ID": [72, "SD75", 74, 74, 71, 72]
})
# Create one-hot encoded variables, also called dummy or indicator variables
df_one_hot = pd.DataFrame(
    enc.fit_transform(df[["ID"]]),
    columns=variables,
    index=df.Patient
)

# Aggregate dummy or one-hot variables, so there's one for each patient
# You may need to alter the aggretaion function
# I chose max because it matched your example
# but perhaps sum might be better (e.g. patient A has two entires for 74, should that be a value of 2 for variable 74?
one_hot_patient = df_one_hot.groupby(level="Patient").agg(max)

# Finally, evaluate your expression
# Create a function to calcualte the output given a data frame
def my_expr(DF):
    out = (DF[71] + DF[72]) \
        * (DF[73] + DF[75] + DF["SD75"]) \
        * (DF[74]+DF[76]+DF["SD76"])
    return out
    
output = one_hot_patient.assign(FinalVal=my_expr)

Result

          71   72   73   74   75   76  SD75  SD76  FinalVal
Patient
A        0.0  1.0  0.0  1.0  0.0  0.0   1.0   0.0       1.0
B        1.0  0.0  0.0  0.0  0.0  0.0   0.0   0.0       0.0
C        0.0  1.0  0.0  0.0  0.0  0.0   0.0   0.0       0.0

Using sub instead of replace should work:

for j in mylist:
    if j in new_dfl:
        exp = re.sub(r'\b{}'.format(j), '1', exp)
    else:
        exp = re.sub(r'\b{}'.format(j), '0', exp)

Another way that would work for this exact scenario is to sort mylist in descending order so the items preceded by SD are iterated before the others.

mylist = re.sub(r'[^\w]', ' ', exp).split()   
mylist.sort(reverse=True)