pyspark

4047 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
How to enable glue logging for AWS Glue script only
user_10620150
•
2
2
1125
pyspark amazon-web-services
Conversion from Pandas data frame to Spark data frame takes huge amount of time
user_121766020
•
2
1
302
pyspark pandas dataframe python
Truncate tables on databricks
user_153113270
•
2
2
1685
pyspark azure-databricks databricks apache-spark azure-sql-database
Convert Integer Column to Date in PySpark
user_131895470
•
2
2
8836
pyspark-dataframes pyspark apache-spark-sql
Check for S3 directory existence in Databricks notebook
user_130148640
•
2
1
853
pyspark databricks amazon-s3
Joining PySpark dataframes with conditional result column
user_27535010
•
2
4
95
pyspark-dataframes pyspark apache-spark-sql apache-spark python
Kafka with Spark 3.0.1 Structured Streaming : ClassException: org.apache.kafka.common.TopicPartition; class invalid for deserialization
user_151705200
•
2
2
316
spark-structured-streaming pyspark apache-kafka avro java
Why does spark application crash with exception java.net.SocketException: Connection reset even after providing sufficient memory & partitioning?
user_71195010
•
3
1
1462
google-cloud-dataproc pyspark apache-spark
How to create a Redshift table using Glue Data Catalog
user_130314120
•
3
1
2066
aws-glue-data-catalog amazon-redshift pyspark etl amazon-web-services
AWS Pyspark Glue job fails at glueContext.write_dynamic_frame.from_jdbc_conf
user_152950770
•
2
0
761
amazon-redshift aws-glue pyspark
Hot Questions