Fetch all the file names from blob storage which came on particular day using python/pyspark

Swati B 0
Viewed 24

I have multiple files in my blob storage , I want to fetch only the file names in form of a list which came on certain day.

for ex: blob strage has all files like

files  Modified
abc    8/22/2022, 1:34:16 AM
bdg    9/12/2022, 2:34:12 AM
hgf    9/12/2022, 5:34:12 AM

i need to fetch all the files which came on 9/12/2022 how to achieve this using python/pyspark?

pyspark python-3.x python
1 Answers

I'm not sure what's file name format, but try this -

import pyspark.sql.functions as F 

spark.read.format(file_format).load(path).withColumn('file_name', F.input_file_name())

Then filter the df by the relevant date.

Assaf Segev0
Related