How to take only 2 data from arraytype column in Spark Scala?
I got the data like val df = spark.sqlContext.sql("select col1, col2 from test_tbl").
I have data like following:
col1 | col2
--- | ---
a | [test1,test2,test3,test4,.....]
b | [a1,a2,a3,a4,a5,.....]
I want to get data like following:
col1| col2
----|----
a | test1,test2
b | a1,a2
When I am doing df.withColumn("test", col("col2").take(5)) it is not working. It give this error:
value take is not a member of org.apache.spark.sql.ColumnName
How can I get the data in above order?