What would be the best way to read data parquet for data range using spark(scala/java) data frame?
The data is partitioned based on the date columns like year, month and day, wildcards can be used and what would be best way?
val dataframe = sqlContext
.read
.parquet("file:///your/path/data=jDD/year=2015/month=10/day={5,6}/*")
Wildcards can also be used to specify a range of months,days:
val dataframe = sqlContext
.read
.parquet("file:///your/path/data=jDD/year=2015/month=[1-5]/day=[5-10]/*")