apache-spark

6201 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
Resume writing of a large dataset after failure using apache spark
user_130962000
•
2
0
38
apache-spark-sql apache-spark
How can I query corrupt-records without caching the data with Pyspark in Azure Databricks?
user_98857860
•
4
0
645
pyspark databricks apache-spark corrupt caching
Why can't I parse all my data when parsing byte string from TFRecrord file written from spark dataframe?
user_57997990
•
2
0
281
tensorflow-datasets tensorflow2.0 pyspark apache-spark python
Get value from Pyspark Column and compare it to a Python dictionary
user_10095080
•
2
1
1066
pyspark apache-spark-sql apache-spark dictionary python
Airflow/Amazon EMR: The VPC/subnet configuration was invalid: Subnet is required : The specified instance type m5.xlarge can only be used in a VPC
user_156046870
•
3
1
923
amazon-emr airflow apache-spark amazon-web-services python
Why Spark applies a broadcast join for a file with size larger than the autoBroadcastJoinThreshold?
user_77700100
•
2
1
417
apache-spark-sql apache-spark
What is the difference between PySpark and Spark?
user_19304020
•
2
1
522
pyspark apache-spark python
Databricks Timestamp format - How to find exact format?
user_95990910
•
2
1
262
pyspark databricks apache-spark
How can I read headers with .createDataFrame()?
user_54914280
•
2
1
208
pyspark apache-spark-sql apache-spark python
Efficient way to find columns that contain ANY null values
user_45640800
•
2
2
1014
pyspark apache-spark
Hot Questions