apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
pyspark dataframe: remove duplicates in an array column
user_34480110
•
4
1
1197
pyspark apache-spark dataframe python
How to query all the versions from delta lake table at once to track changes made to a specific ID
user_99028130
•
3
2
2994
delta-lake apache-spark-sql apache-spark
Spark Cassandra Write UDT With Case-Sensitive Names Fails
user_49016880
•
2
1
168
spark-cassandra-connector datastax apache-spark cassandra
Neo4j/Apache Spark task is not serializable
user_133144950
•
2
1
33
apache-spark neo4j
How can missing columns be added as null while read a nested JSON using pyspark and a predefined struct schema
user_52105600
•
3
1
1151
apache-spark python json
While Reading CSV, last column is coming as Null in Spark, Scala
user_117975650
•
3
2
534
pyspark apache-spark dataframe csv scala
org.postgresql.util.PSQLException: SSL error: Received fatal alert: handshake_failure while writing from Azure Databricks to Azure Postgres Citus
user_58883960
•
2
1
3315
azure-postgresql pyspark azure-databricks apache-spark azure
In Pyspark Structured Streaming, how can I discard already generated output before writing to Kafka?
user_63851050
•
2
0
67
spark-streaming spark-structured-streaming pyspark apache-spark
How to get AWS EMR cluster id and step id from inside the spark application step submitted
user_28150060
•
2
3
1617
amazon-emr apache-spark amazon-web-services scala
I know how to do an orderBy("a", "b"...), a groupBy on a dataset. I need to perform calculations and work on each subset of records, idependently
user_28271810
•
3
2
245
apache-spark java
Hot Questions