I have some code where I read in many parquet tables via a directory and wildcard, like this:
df = sqlContext.read.load("some_dir/*")
Is there some way I can get the source file for each row in the resulting DataFrame, df?
I have some code where I read in many parquet tables via a directory and wildcard, like this:
df = sqlContext.read.load("some_dir/*")
Is there some way I can get the source file for each row in the resulting DataFrame, df?