apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
FileNotFoundException on _temporary/0 directory when saving Parquet files
user_41268430
•
3
1
1891
hdfs azure-blob-storage apache-spark hadoop python
Spark input size when reading Parquet from S3 is 2x higher than from local FS
user_44406150
•
3
0
88
parquet apache-spark amazon-s3
Create new columns in Spark dataframe based on a hundred column pairs
user_176925310
•
2
1
131
apache-spark dataframe scala
Convert struct to map in Spark SQL
user_16808260
•
3
2
1399
apache-spark-sql apache-spark
Provider com.google.cloud.spark.bigquery.BigQueryRelationProvider could not be instantiated while reading from bigquery in Jupyter lab
user_151130290
•
1
1
2062
pyspark apache-spark google-bigquery scala
Tuple manipulation in PySpark with lambda, python
user_41334640
•
2
1
149
rdd pyspark apache-spark tuples python
How create parquet table in scala?
user_159859930
•
2
1
131
parquet apache-spark hive hadoop scala
getting Py4JJavaError while using apache spark big query connector
user_151130290
•
1
1
661
apache-spark google-bigquery scala python
How to write custom dataframe to eventhub from ADLS
user_176906990
•
3
1
307
azure-eventhub apache-spark-sql apache-spark azure scala
Spark Structured Streaming program that reads from non-empty Kafka topic (starting from earliest) triggers batches locally, but not on EMR cluster
user_12243630
•
5
2
400
amazon-emr spark-structured-streaming apache-kafka apache-spark
Hot Questions