I want to run grades example proposed by apache beam go sdk using spark runner on a spark cluster with one master and two slaves (spark2.4.5 version ). I start the job runner using ./gradlew :runners:spark:2:job-server:runShadow -PsparkMasterUrl=spark://localhost:7077 and run it using the following command : grades -runner=spark -endpoint=localhost:8099 -job_name=gradetest
However I get the following error.
2021/10/08 11:42:34 (): org.apache.beam.sdk.Pipeline$PipelineExecutionException: java.lang.NoSuchMethodError: com.fasterxml.jackson.databind.type.TypeBindings.emptyBindings()Lcom/fasterxml/jackson/databind/type/TypeBindings;
at org.apache.beam.runners.spark.SparkPipelineResult.beamExceptionFrom(SparkPipelineResult.java:73)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:104)
at org.apache.beam.runners.spark.SparkPipelineResult.waitUntilFinish(SparkPipelineResult.java:92)
at org.apache.beam.runners.spark.SparkPipelineRunner.run(SparkPipelineRunner.java:199)
I assume that the problem can be from the spark job runner because i test another code using python and I get the same error. Any help would be gratefully appreciated