I have a notebook a.pynb which has the function for read statement of a parquet file.
I am using a.pynb in another notebook b.pynb and in this new notebook, i am calling a function of a.pynb to read this parquet file and create a sqltable. But it always fails with
Error: global name sqlContext is not defined,
When it is defined in both the notebooks.
The exact code :
a.pynb ( Utils)
sc = SparkContext.getOrCreate()
sqlContext = SQLContext(sc)
def parquet_read(file_name):
df = sqlContext.read.parquet(file_name+"*.parquet")
return df
In b.pynb I have used this function
import nbimporter
import a as commonUtils
reload(commonUtils)
sc = SparkContext.getOrCreate()
sqlContext = SQLContext(sc)
df2 = commonUtils.parquet_read("abc")
It always fails with
Error: global name sqlContext is not defined,
When it is defined in both the notebooks.