apache-spark

apache-spark

436 Questions
Newest Active Unanswered Frequent
Score
View
Card Compact
pyspark calculate custom metric on grouped data
user142973390
•
1
0
49
pyspark-pandas large-data pandas-udf pyspark apache-spark
Using Spark I am injecting data From S3 to Hbase. But, Can I do it with Sqoop? If yes then why? and If no then why?
Sushil Nyayadhish0
•
-1
0
22
sqoop hadoop amazon-s3 apache-spark amazon-web-services
Exception in thread "main" java.io.FileNotFoundException: File does not exist: hdfs:
Juanpita400
•
0
0
53
hdfs hadoop apache-spark
while executing pyspark datafram giving error
userxdJ810
•
0
1
37
pyspark apache-spark apache-spark-sql python
Does using PySpark "functions.expr()" have a performance impact on query?
QPeiran0
•
1
1
39
performance dataframe pyspark apache-spark apache-spark-sql
Subtract columns in two dataframes to get differences in Spark Scala
jmarco0
•
2
1
35
dataframe apache-spark scala
Killing the spark.sql
mayur benodekar0
•
0
0
36
apache-spark apache-spark-sql scala
Error when running graphframes in google colab
Tytire Recubans0
•
1
1
30
graphframes google-colaboratory pyspark apache-spark
In pyspark, how to groupBy and collect a list of all distinct structs contained in an array column
Tytire Recubans0
•
0
1
29
pyspark apache-spark apache-spark-sql
In Apache Beam's SparkRunner, how does the DOCKER environment_type affect an existing Spark cluster?
cozos0
•
0
1
37
apache-beam-internals apache-beam google-cloud-dataproc google-cloud-platform apache-spark
Hot Questions