we had some old code using org.apache.parquet.hadoop.api.WriteSupport API to write Parquet formatted file, and we start to use Apache Spark to do the same thing.
Those two ways can successfully generate Parquet files with same input data, and the output data are almost identical. However, the output file size is quite different.
The one generated by WriteSupport is 2G-ish, whereas the one generated by Spark is 5.5G-ish. I compared the schema, they are same, is there any area I can further look into?
Btw, the WriteSupport has parquet-mr version 1.8.0; Spark one has 1.10.0.