I know similar questions are already posted and answered and referred them before posting this.
I am using the
EMR emr-6.6.0
spark-snowflake_2.13-2.10.1-spark_3.2.jar
snowflake-jdbc-3.13.14.jar
But I still getting the error as title. I do understand this fact is Spark 3.2 has some compatibility issues. For which I referred Here. However despite all I am still getting the same error.
py4j.protocol.Py4JJavaError: An error occurred while calling o124.save.
Above 2 jars I kept in S3 bucket and using below commands to call them .
"sparkSubmitParameters": "--conf spark.jars=s3://mydir/sqljdbc42.jar,s3://mydir/snowflake-jdbc-3.13.14.jar,s3://mydir/spark-snowflake_2.13-2.10.1-spark_3.2.jar --conf spark.submit.pyFiles=s3://mydir/ ......
It is loading s3://mydir/sqljdbc42.jar and doing his job so no issue of not loading the jar.
Please help me to resolve this.