Adding data to scala WrappedArray in Java

user_3311616 0
Viewed 194

I have the below scala Row which I am getting as part of a JavaRdd.map() call in Java code.

["A",4,[WrappedArray(["B"],["C"]),WrappedArray(["D"])]]

Schema :

{"type":"struct","fields":[{"name":"id","nullable":false,"type":"string","metadata":{}},{"name":"mid","nullable":false,"type":"long","metadata":{}},{"name":"cr","nullable":false,"type":{"type":"struct","fields":[{"name":"psd","type":{"type":"array","elementType":{"type":"struct","fields":[{"name":"cid","type":"string","nullable":true,"metadata":{}}]},"containsNull":true},"nullable":true,"metadata":{}},{"name":"pqd","type":{"type":"array","elementType":{"type":"struct","fields":[{"name":"cid","type":"string","nullable":true,"metadata":{}}]},"containsNull":true},"nullable":true,"metadata":{}}]},"metadata":{}}]}

I want to create a new Row with the contents of the above row just adding string "X" to the internal elements of the WrappedArray.

["A",4,[WrappedArray(["X", "B"],["X", "C"]),WrappedArray(["X", D"])]]

Schema :

{"type":"struct","fields":[{"name":"id","nullable":false,"type":"string","metadata":{}},{"name":"mid","nullable":false,"type":"long","metadata":{}},{"name":"cr","nullable":false,"type":{"type":"struct","fields":[{"name":"psd","type":{"type":"array","elementType":{"type":"struct","fields":[{"name":"cid","type":"string","nullable":true,"metadata":{}},{"name":"renamed","type":"string","nullable":true,"metadata":{}}]},"containsNull":true},"nullable":true,"metadata":{}},{"name":"pqd","type":{"type":"array","elementType":{"type":"struct","fields":[{"name":"cid","type":"string","nullable":true,"metadata":{}},{"name":"renamed","type":"string","nullable":true,"metadata":{}}]},"containsNull":true},"nullable":true,"metadata":{}}]},"metadata":{}}]}

How do you create the new Row by adding extra data to the existing struct inside the WrappedArray.

apache-spark-sql apache-spark scala java
1 Answers

WrappedArray is an array-like type in Scala. You can create a new WrappedArray by appending an element like this, and then you can update the object:

val arr = new WrappedArray.ofRef(Array("a", "b", "c"))
val newArr = arr :+ "d"
user_3591200
Related