I am using Java 8 with Spark v2.4.1.
I am trying to add Map using Spark function called typedLit. But getting compilation errors. How can I do it in Java API?
Below is the scenario:
Map<Integer,Integer> lookup_map= new HashMap<>();
lookup_map.put(1,11);
lookup_map.put(2,21);
lookup_map.put(3,31);
lookup_map.put(4,41);
lookup_map.put(5,51);
JavaSparkContext javaSparkContext = JavaSparkContext.fromSparkContext(sparkSession.sparkContext());
Column typedMapCol = functions.typedLit(lookUpScoreHm, Map<Encoders.INT(), Encoders.INT()> );
// this is not correct and giving error in at typedLit.
Dataset<Row> resultDs= dataDs
.withColumn("map_col", typedMapCol)
How to define functions.typedLit in Java 8?