apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Databricks - How to join a table with IDs contained in a column of type struct<array<string>>
user_63406090
•
1
1
358
delta-lake pyspark databricks apache-spark-sql apache-spark
Is there a way to pass an additional/extra parameter in `foreachBatch` function in Kafka Streaming?
user_71195010
•
1
1
399
spark-streaming apache-kafka apache-spark
How to merge a spark dataframe with hive table on Databricks Deltalake?
user_71195010
•
1
1
806
delta-lake databricks apache-spark
avoid pivot when preprocessing data for vector assembler
user_34764630
•
1
0
81
apache-spark-mllib apache-spark-ml pyspark apache-spark
Databricks Stream to Batch process
user_151712700
•
4
0
356
aws-databricks azure-databricks databricks apache-spark azure
Spark JDBC: Incorrect syntax in spark.read
user_149341810
•
3
3
842
pyspark databricks apache-spark jdbc
Order of Execution in Spark DAG
user_102230830
•
1
1
71
apache-spark scala
Why I'm getting CSVHeaderChecker:69 - CSV header does not conform to the schema.?
user_17590630
•
1
1
1094
spark-csv spark-structured-streaming apache-spark
inferSchema=true isn't working for csv file reading n Spark Structured Streaming
user_17590630
•
3
2
1269
spark-csv spark-structured-streaming apache-spark scala
Create a new dataframe (with different schema) from selected information from another dataframe
user_144766900
•
1
2
262
apache-spark-sql apache-spark scalability scala java
Hot Questions