Skip file if value is not in data using python

Viewed 39

With my current code, I am trying to skip a csv file if it does not contain a value within the actual data that I am looking for.

basically if it has "PROD_NAME" as a column, then it looks for that string and replaces it with the second string in that statement, but the first file in my folder does not have this column name and so the script fails. I've looked into ways to skip but have only seen ways to skip based on the filename itself and not the data within a file not having the correct information. Any help would be appreciated. Thanks!

def worker(files):
    filenames = glob.glob(dest_dir + '\\*.csv')
    for filename in filenames:
            
        my_file = Path(os.path.join(dest_dir, filename))
        
        #read header
        with open(filename) as f:
            read_data = f.read()
        header = read_data[:read_data.find('!1')]
        idx = header.find('\n')
        

        # read data
        df1 = pd.read_csv(filename, skiprows=1, encoding='ISO-8859-1', nrows=1) # read column header only - to get the list of columns
        dtypes = {}
        for col in df1.columns:# make all columns text, to avoid formatting errors
            dtypes[col] = 'str'
        df1 = pd.read_csv(filename, dtype=dtypes, skiprows=1, encoding='ISO-8859-1', quotechar="'", delimiter='\t')
        
        
        df1.loc[df1['PROD_NAME'].str.contains('NA_NRF'), 'PROD_NAME'] = 'FA_GUAR'
        file_count += 1 # count the fil
            
worker(files)
1 Answers

Could you just add an if statement before your transformation

if 'PROD_NAME' in df1.columns:            
    df1.loc[df1['PROD_NAME'].str.contains('NA_NRF'), 'PROD_NAME'] = 'FA_GUAR'
    
file_count += 1 # count the fil
Related