How to check if all the required columns are present in excel using python/pyspark

Swati B 0
Viewed 42

I have multiple Excel files which I'm receiving in my blob storage. I have certain columns which I need for sure in each of those excel

for ex;
excel1= ['a','b','c']
excel2=['d','e','f']

I want to fetch only the column names from all these excel and check whether the required columns are present or not and assert if not present. how to achieve this using pyspark?

azure-databricks pyspark python
1 Answers

See if this helps you:

listColumns=df.columns
"column_name"  in listColumns

You can read more here.

snowmoss0
Related