Databricks fails to create StructType / Schema when used against case class installed as Jar file

Viewed 62

I am using ScalaReflection to create schema from Case Classes. I have installed the jar containing the case classes in the databricks cluster and When I invoke the following command ScalaReflection.schemaFor[TESTCaseClass].dataType.asInstanceOf[StructType] I get the following error AnnotationFormatError: Duplicate annotation for class: interface scala.reflect.ScalaLongSignature: @scala.reflect.ScalaLongSignature

Below is the code that I am doing

import org.apache.spark.sql.catalyst.ScalaReflection
import org.apache.spark.sql.types.StructType 
case class TEST(
                 VAR1: String,
                 VAR2: String
               )
def TestSchema: StructType = ScalaReflection.schemaFor[TEST].dataType.asInstanceOf[StructType]

Scala version : 2.12.10
Spark Version : 3.1.1
Databricks cluster runtime : 8.3

I compile the class to .class file and it fails in databricks while invoking from databricks cell with following error

AnnotationFormatError: Duplicate annotation for class: interface scala.reflect.ScalaLongSignature: @scala.reflect.ScalaLongSignature
0 Answers
Related