I am running a EMR step(spark submit) with fat shaded jar but old version of libraries from EMR gets picked up causing NoSuchMethod exceptions
I tried shading jar and set below but nothing seem to work. Old versions on EMR are loaded first
spark.driver.userClassPathFirst & spark.executor.userClassPathFirst
How to ensure classes in app jar get precedence over EMR libraries