I am working with Nested json, using scala and need to replace the - in column names with _.
Schema of json:
|-- a-type: struct (nullable = true)
| |-- x-Type: array (nullable = true)
| | |-- element: string (containsNull = true)
| |-- part: array (nullable = true)
| | |-- element: struct (containsNull = true)
| | | |-- x-Type: array (nullable = true)
| | | | |-- element: string (containsNull = true)
| | | |-- Length: long (nullable = true)
| | | |-- Order: long (nullable = true)
| | | |-- y-Name: string (nullable = true)
| | | |-- Payload-Text: string (nullable = true)
| |-- Date: string (nullable = true)
I am using below code which only works at first level. However, I have to replace - with _ at all levels. Any help is really appreciated.
Code used currently:
scJsonDF.columns.foreach { col =>
println(col + " after column replace " + col.replaceAll("-", ""))
scJsonDFCorrectedCols = scJsonDFCorrectedCols.withColumnRenamed(col, col.replaceAll("-", "")
)
}
I am looking for a dynamic solution as there are different structures available.