apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Is it possible to get the attempt number of a running Spark Task?
user_68772130
•
2
0
79
pyspark apache-spark
how to read parquet files in pyspark as per the defined schema before read?
user_82360290
•
2
1
625
pyspark apache-spark
How do I specify output log file during spark submit
user_100255040
•
3
1
763
apache-spark log4j logging
Neither pyarrow nor fastparquet able to read vector output from spark correctly
user_149819570
•
2
0
443
fastparquet parquet pyarrow pandas apache-spark
How I can get the same result using iloc in Pandas in PySpark?
Admin1
•
2
1
403
pyspark apache-spark-sql apache-spark
Filtering a column with an empty array in Pyspark
user_4270690
•
3
2
1284
pyspark apache-spark-sql apache-spark python
Debug Pyspark on EMR using Pycharm
user_96013350
•
7
1
835
amazon-emr pyspark apache-spark pycharm amazon-web-services
delta lake in databricks - a consistent "view" of just the last half hour of a stream
user_6902040
•
2
1
706
delta-lake databricks apache-spark
Delta lake in databricks - creating a table for existing storage
user_6902040
•
2
1
7201
delta-lake databricks apache-spark
Structured streaming output - compacting with OPTIMIZE without breaking outgoing read stream order guarantees
user_6902040
•
2
1
696
delta-lake spark-streaming databricks apache-spark
Hot Questions