parquet

503 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Apache Spark - Write Parquet Files to S3 with both Dynamic Partition Overwrite and S3 Committer
user_157411190
•
2
1
836
parquet pyspark apache-spark amazon-web-services amazon-s3
Using Spark 3.2 to ingest IoT data into delta lake continuously
user_2697390
•
2
1
103
delta-lake parquet apache-spark scala java
Spark partitioning of related data into row groups
user_6471510
•
1
1
421
parquet apache-spark
Pyspark read and combine many parquet files efficiently
user_55831000
•
1
0
708
parquet pyspark apache-spark-sql python
My S3 bucket has more than 100k parquet files. What is the best way to programmatically merge all the parquet files and make one big parquet
user_87084790
•
1
1
411
parquet pandas apache-spark dataframe amazon-s3
How to read parquet files in pyspark from s3 bucket whose path is partially unpredictable?
user_12807260
•
1
2
1036
parquet apache-spark wildcard amazon-s3
How to convert parquet to json
user_166036290
•
2
0
1856
fastparquet parquet python-3.x amazon-s3 json
Reading single parquet-partition with single file results in DataFrame with more partitions
user_37576720
•
2
1
923
parquet pyspark apache-spark python
Spark input size when reading Parquet from S3 is 2x higher than from local FS
user_44406150
•
3
0
88
parquet apache-spark amazon-s3
AWS Athena isn't using partitions discovered by Glue crawler, while Redshift Spectrum is
user_45699230
•
3
0
340
amazon-redshift parquet amazon-athena aws-glue partitioning
Hot Questions