Kotlin (and scala): No parameter with index 0-0 (name=reverser$module$1 access=16) in method scala.collection.TraversableOnce.reverser$2

user_5060325 0
Viewed 423

I am using spark 3.0.1 in my kotlin project. Compilation fails with the following error:

e: org.jetbrains.kotlin.util.KotlinFrontEndException: Exception while analyzing expression at (51,45) in /home/user/project/src/main/kotlin/ModelBuilder.kt
...
Caused by: java.lang.IllegalStateException: No parameter with index 0-0 (name=reverser$module$1 access=16) in method scala.collection.TraversableOnce.reverser$2
        at org.jetbrains.kotlin.load.java.structure.impl.classFiles.AnnotationsAndParameterCollectorMethodVisitor.visitParameter(Annotations.kt:48)
        at org.jetbrains.org.objectweb.asm.ClassReader.readMethod(ClassReader.java:1149)
        at org.jetbrains.org.objectweb.asm.ClassReader.accept(ClassReader.java:680)
        at org.jetbrains.org.objectweb.asm.ClassReader.accept(ClassReader.java:392)
        at org.jetbrains.kotlin.load.java.structure.impl.classFiles.BinaryJavaClass.<init>(BinaryJavaClass.kt:77)
        at org.jetbrains.kotlin.load.java.structure.impl.classFiles.BinaryJavaClass.<init>(BinaryJavaClass.kt:40)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCliJavaFileManagerImpl.findClass(KotlinCliJavaFileManagerImpl.kt:115)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCliJavaFileManagerImpl.findClass(KotlinCliJavaFileManagerImpl.kt:85)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCliJavaFileManagerImpl$findClass$$inlined$getOrPut$lambda$1.invoke(KotlinCliJavaFileManagerImpl.kt:113)
        at org.jetbrains.kotlin.cli.jvm.compiler.KotlinCliJavaFileManagerImpl$findClass$$inlined$getOrPut$lambda$1.invoke(KotlinCliJavaFileManagerImpl.kt:48)
        at org.jetbrains.kotlin.load.java.structure.impl.classFiles.ClassifierResolutionContext.resolveClass(ClassifierResolutionContext.kt:60)
        at org.jetbrains.kotlin.load.java.structure.impl.classFiles.ClassifierResolutionContext.resolveByInternalName$frontend_java(ClassifierResolutionContext.kt:101)
        at org.jetbrains.kotlin.load.java.structure.impl.classFiles.BinaryClassSignatureParser$parseParameterizedClassRefSignature$1.invoke(BinaryClassSignatureParser.kt:141)

I've cleaned/rebuilt the project several times, removed the build directory and tried building from the command line with gradle.

The code where this happens:

    val data = listOf(...)
    val schema = StructType(arrayOf(
            StructField("label", DataTypes.DoubleType, false, Metadata.empty()),
            StructField("sentence", DataTypes.StringType, false, Metadata.empty())
    ))

    val dataframe = spark.createDataFrame(data, schema) // <- offending line.

Was using kotlin version 1.4.0, upgraded to 1.4.10 without any change, still same error.

Looks like this bug (and this) already reported to JetBrains, but is it really not possible to use spark 3 (local mode) in kotlin 1.4?

apache-spark kotlin scala
2 Answers

I managed to get it working with Spring Boot (2.3.5) by adding the following to the dependencyManagement:

dependencies {
    dependencySet("org.scala-lang:2.12.10") {
        entry("scala-library")
    }
}

This will downgrade the scala-library jar from 2.12.12 to 2.12.10 version, which is the same version of the scala-reflect jar in my project. I'm also using Kotlin 1.4.10

user_112468890
user_8129060
Related