How to read set of files with different filenames into databricks notebook from blob storage

Question

I want to read all the Personal_details_info files into databricks notebook to process the data which are stored in the blob storage.

Personal_detail_Info_20220908000234
Personal_detail_Info_20220907000023
Personal_detail_Info_20220906000356

Customer_details_Info_20220908000342
Customer_details_Info_20220907000324
Customer_details_Info_20220906000123

BhanunagasaiVamsi · Answer

If you want to read Personal_details_info file. Please follow this syntax :

You can check your file location:

Ref1

Code:

Use * at the end of Personal_detail_Info_* file location.

df = spark.read.option("header",True).format("csv").load(f"//Personal_detail_Info_*").csv")
display(df)

enter image description here