Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources error

user_13559009 0
Viewed 92

I am new in spark. I am going to start spark by using

/opt/spark/bin/spark-submit --jars mariadb-java-client-3.0.5.jar --master spark://neem-2:7077 sparksql.py

But I get this error and I stuck with this, I would really appreciate that if you can help me we that. I know it is about worker but I really don't know how to overcome this problem.

WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
spark-submit spark-streaming pyspark apache-spark
1 Answers

Check yarn timeline server URL, and confirm if you have the enough resources available to run the given job. It could be possible that you have other jobs running and it is taking up all the resources already. Try increasing the cluster size if necessary.

user_72082590
Related