apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
PySpark: Numpy memory not being released in executor map-partition function (memory leak)
user_11103280
•
13
2
1156
pyspark apache-spark memory-leaks numpy python
How to set up autocomplete for Databricks notebooks
user_28770290
•
4
2
5684
databricks apache-spark editing autocomplete intellij-idea
drop column in a table/view using spark sql only
user_104616280
•
4
2
8700
apache-spark-sql apache-spark string-interpolation
Spark development process with Python and IDE
user_105900060
•
3
1
5690
apache-spark pycharm python
Submitting Spark job to Amazon EMR
user_22927910
•
4
1
4542
amazon-emr apache-spark
How to list S3 objects in parallel in PySpark using flatMap()?
user_27959410
•
3
1
1689
pyspark boto3 apache-spark-sql apache-spark amazon-s3
org.apache.spark.sql.AnalysisException: 'write' can not be called on streaming Dataset/DataFrame
user_10334220
•
5
1
4098
spark-streaming spark-structured-streaming apache-spark
Deduplicate Rows with reduceGroups won't find Encoder[Row]
user_5997280
•
3
0
224
apache-spark scala
Processing JSON in spark - different schemas in different files
user_73652650
•
3
1
1417
apache-spark schema json
Mysterious 'pyarrow.lib.ArrowInvalid: Floating point value truncated' ERROR when use toPandas() on a DataFrame in pyspark
user_74274710
•
4
2
6073
apache-arrow pyspark pyarrow apache-spark-sql apache-spark
Hot Questions