apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Spark batch processing on driver
user_140748580
•
2
1
82
bigdata apache-spark-sql apache-spark java
How to get Kafka header's value to Spark Dataset as a single column?
user_17590630
•
2
1
746
spark-structured-streaming apache-kafka apache-spark scala
A Persisted Spark DataFrame is computed twice
user_78082040
•
2
0
247
pyspark apache-spark-sql apache-spark persist caching
Understanding --archive in dataproc pyspark
user_80896950
•
2
1
281
dataproc google-cloud-dataproc pyspark apache-spark
Spark submit fails: No such file or directory: '/opt/spark/python/pyspark/./bin/spark-submit'
user_179759140
•
2
0
214
pyspark apache-spark
Pyspark variable has datatype decimal(6,-12). df.dtypes and df.columns gives error ValueError: Could not parse datatype: decimal(6,-12)
user_179752680
•
2
1
233
pyspark apache-spark dataframe python
Split Pyspark dataframe into multiple json files based on a particular column data?
user_179638650
•
2
1
263
pyspark apache-spark dataframe python json
PySpark virtual environment archive on S3
user_46019310
•
3
0
312
livy amazon-emr pyspark apache-spark
Visualization over delta table residing in S3
user_179704970
•
2
0
66
delta-lake apache-spark amazon-s3
JVM Crash with sun.misc.Unsafe::GetInt when write Spark Dataframe in parquet format (SPARC Solaris)
user_114541890
•
2
0
116
sparc apache-spark jvm-crash solaris hadoop
Hot Questions