I have EMR cluster and notebook. Since I want to work with PySpark, I changed my kernel to PYSPARK. Before
import pyspark
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-2-49d7c4e178f8> in <module>
----> 1 import pyspark
ModuleNotFoundError: No module named 'pyspark'
After the kernel was changed to PySpark
import pyspark
Error in callback <bound method WidgetKeeper.check_if_monitorable of <awseditorssparkmonitoringwidget.widgetkeeper.WidgetKeeper object at 0x7f37e11359d0>> (for pre_run_cell):
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
/opt/conda/lib/python3.7/site-packages/awseditorssparkmonitoringwidget-1.0-py3.7.egg/awseditorssparkmonitoringwidget/widgetkeeper.py in check_if_monitorable(self, info)
AttributeError: 'WidgetKeeper' object has no attribute 'request_target_endpoint'
The code failed because of a fatal error:
'notebookId'.
Some things to try:
a) Make sure Spark has enough available resources for Jupyter to create a Spark context.
b) Contact your Jupyter administrator to make sure the Spark magics library is configured correctly.
c) Restart the kernel.
Where to find more about notebookId? How to check Resources issue?