Efficient way to check if an attribute name exists in the master list?

user_9787406 0
Viewed 31

Hello I am trying to check if column names in the uploaded dataframes have any sensitive elements. Like so...

uploaded_csv_files = [['emp_ssn','emp_phone','emp_name'],['worker_phone', 'worker_credit_card','worker_name'],['vendor_ssn','vendor_credit_card','vendor_phone']]

sensitive_list_master = ['ssn', 'credit', 'phone']

 

for df in uploaded_csv_files:

  for column in df:

    for each in column.split("_"):

      if each in sensitive_list:

        print(f"{column} has a sensitive element")

As you can see I am using nested for loops which could become computationally inefficient when a lot of columns are involved. Is there a better way to do this ?

iterator python loops
1 Answers

Assuming you must check each column, this is slightly more efficient:

for df in uploaded_csv_files:
  for column in df:
    for s in sensitive_list:
      if s in column:
        print(f"{column} has a sensitive element")

The fact that you have to check each column means that there is a linear amount of work to be done. So you're stuck with that constraint.

user_42252290
Related