Upgraded the spark version, and during spark jobs encountering java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V

We recently made an upgrade from Spark 2.4.2 to 2.4.5 for our ETL project.

After deploying the changes, and running the job I am seeing the following error:

    Exception in thread "main" java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.spark.deploy.worker.DriverWrapper$.main(DriverWrapper.scala:65)
    at org.apache.spark.deploy.worker.DriverWrapper.main(DriverWrapper.scala)
Caused by: java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V
    at com.advisory.pic.etl.utils.OracleDialect$.<init>(OracleDialect.scala:12)
    at com.advisory.pic.etl.utils.OracleDialect$.<clinit>(OracleDialect.scala)
    at com.advisory.pic.etl.drivers.BaseDriver.$init$(BaseDriver.scala:19)
    at com.advisory.pic.etl.drivers.PASLoadDriver$.<init>(PASLoadDriver.scala:19)
    at com.advisory.pic.etl.drivers.PASLoadDriver$.<clinit>(PASLoadDriver.scala)
    at com.advisory.pic.etl.drivers.PASLoadDriver.main(PASLoadDriver.scala)
    ... 6 more

I read online that it could be possible due to library version mismatch, but I could not find any such violations in our build.gradle, except for testImplementation, which I upgraded to correct version, but I doubt this is the root cause of the issue.

Following is a snippet of dependencies in build.gradle file.

dependencies {
    def hadoopClientVersion = '2.7.1'
    def hadoopCommonsVersion = '2.7.1'
    def sparkVersion = '2.4.5'
    def sparkTestingVersion = '2.4.5'



    provided group: 'org.apache.hadoop', name: 'hadoop-client', version: hadoopClientVersion
    provided group: 'org.apache.hadoop', name: 'hadoop-common', version: hadoopCommonsVersion

    implementation("org.apache.spark:spark-sql_2.12:$sparkVersion"){
        //excluding is causing issues when running through IDE - as sparkLibraries are not available at run-time
        //We can comment while going for deployment if we face jar conflict issues
        //exclude module: 'spark-core_2.10'
        //exclude module: 'spark-catalyst_2.10'
    }
    implementation group: 'org.apache.spark', name: 'spark-core_2.12', version: sparkVersion
    testImplementation group: 'org.apache.spark', name: 'spark-core_2.12', version: sparkVersion

    // spark -sql with avro
    implementation("com.databricks:spark-avro_2.11:4.0.0")

    // joda-time
    implementation 'com.github.nscala-time:nscala-time_2.12:2.22.0'

    //configuration object
    implementation group: 'com.typesafe', name: 'config', version: '1.2.1'

    implementation "ch.qos.logback:logback-classic:1.1.3"
    implementation "org.slf4j:log4j-over-slf4j:1.7.13"



    // Libraries needed for scala api
    implementation 'org.scala-lang:scala-library:2.12.0'
    implementation 'org.scala-lang:scala-compiler:2.12.0'


    testImplementation 'org.scalatest:scalatest_2.12:3.0.5'

    implementation 'com.oracle:ojdbc7:12.1.0.1'

    testImplementation group: 'com.h2database', name: 'h2', version: '1.4.196'

    testImplementation 'com.holdenkarau:spark-testing-base_2.12:' + sparkTestingVersion + '_0.12.0'
    itestCompile 'org.scala-lang:scala-library:2.12.0'
    itestCompile 'org.scalatest:scalatest_2.12:3.0.5'
    testImplementation 'org.scalamock:scalamock_2.12:4.3.0'
}

Any suggestions on why the above issue is happening, and how can I verify the version mismatch?