apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Why write from databricks spark notebook ( hadoop fileUtils) to DBFS mount location is 13 times slower than write to DBFS Root location?
user_89180510
•
1
1
172
azure-databricks databricks apache-spark hadoop
pyspark.sql.utils.ParseException: mismatched input '#' expecting {<EOF>
user_181460070
•
2
1
451
amazon-dynamodb aws-glue pyspark apache-spark-sql apache-spark
reading a datetime column as stringType in pyspark and converting it to datetime giving null records
user_152833640
•
1
1
42
pyspark apache-spark python
Pyspark check if value in dictionary or map using when() otherwise()
user_159223090
•
2
1
283
pyspark apache-spark python
RDD order after SVD
user_102156950
•
1
0
30
svd rdd apache-spark java
Custom Payload class in Python for precombine and combineAndGet in Apache Hudi And Pyspark
user_27276640
•
2
0
140
apache-hudi pyspark apache-spark-sql apache-spark
How to explode a string column based on specific delimiter in Spark Scala
user_56178670
•
1
1
102
apache-spark scala
PySpark Structured Streaming: trigger once not working with Kafka
user_154110760
•
2
1
237
spark-streaming pyspark apache-kafka apache-spark
Group by and aggregate on a column with array in PySpark
user_88326410
•
1
2
728
pyspark apache-spark-sql apache-spark aggregate-functions arrays
AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] when using Hive warehouse
user_41268430
•
2
1
1690
azure-hdinsight apache-spark kerberos hive
Hot Questions