unable to mock GlueContext from aws glue
I have these lines of code
sc = SparkContext()
gluecontext = GlueContext(sc)
spark = gluecontext.spark_session
How to mock gluecontext using python unittest?
How to mock in a python unittest a library not installed locally?
One way is to download Glue libs and Spark to the root of your project (or configure as you like)
wget https://github.com/awslabs/aws-glue-libs/archive/glue-1.0.zip
wget https://aws-glue-etl-artifacts.s3.amazonaws.com/glue-1.0/spark-2.4.3-bin-hadoop2.8.tgz
unzip glue-1.0.zip -d $PROJECT_ROOT
tar -xf spark-2.4.3-bin-hadoop2.8.tgz -C $PROJECT_ROOT
export SPARK_HOME=$PROJECT_ROOT/spark-2.4.3-bin-spark-2.4.3-bin-hadoop2.8
Then simply mock gluecontext
from mock import patch
class Test(unittest.TestCase):
@patch('awsglue.context.GlueContext')
@patch('awsglue.utils.getResolvedOptions', side_effect=mock_get_resolved_options)
def test_method(self, mock_resolve_options, mock_glue_context):
<your code>
Submit tests locally
$PROJECT_ROOT/aws-glue-libs-glue-1.0/bin/gluepytest $PROJECT_ROOT/tests/