How to use Spark typedLit function in Java

Viewed 534

I am using Java 8 with Spark v2.4.1.

I am trying to add Map using Spark function called typedLit. But getting compilation errors. How can I do it in Java API?

Below is the scenario:

    Map<Integer,Integer> lookup_map= new HashMap<>();
    lookup_map.put(1,11);
    lookup_map.put(2,21);
    lookup_map.put(3,31);
    lookup_map.put(4,41);
    lookup_map.put(5,51);
    
    JavaSparkContext javaSparkContext = JavaSparkContext.fromSparkContext(sparkSession.sparkContext());
   
   Column typedMapCol = functions.typedLit(lookUpScoreHm, Map<Encoders.INT(), Encoders.INT()>  );  
    // this is not correct and giving error in at typedLit.
    
    
        Dataset<Row> resultDs= dataDs
           .withColumn("map_col", typedMapCol)

How to define functions.typedLit in Java 8?

0 Answers
Related