Parquet format store bad records separately using pyspark

Viewed 163

I have been trying to load the parquet files but like to find if any bad records with in the parquet files and then store those records separately. This is quite possible for csv format (Pyspark has built in mode i.e. permissive) but I am not sure how this can be done for parquet. If anyone have come across this issue then please let me know.

0 Answers
Related