I am trying to read a UTF16-LE JSON file with a "document per line". I During the read file I get the following exception. I don't see a "lineSep" option in the portal. How does one set this?
Operation on target dataflow1 failed:
{"StatusCode":"DFExecutorUserError","Message":"Job failed due to reason: at Source 'source1': java.lang.IllegalArgumentException: requirement failed: The lineSep option must be specified for the UTF-16LE encoding","Details":"java.lang.IllegalArgumentException: requirement failed: The lineSep option must be specified for the UTF-16LE encoding\n\tat scala.Predef$.require(Predef.scala:224)\n\tat org.apache.spark.sql.catalyst.json.JSONOptionsInRead.checkedEncoding(JSONOptions.scala:153)\n\tat org.apache.spark.sql.catalyst.json.JSONOptions$$anonfun$34.apply(JSONOptions.scala:109)\n\tat org.apache.spark.sql.catalyst.json.JSONOptions$$anonfun$34.apply(JSONOptions.scala:109)\n\tat scala.Option.map(Option.scala:146)\n\tat org.apache.spark.sql.catalyst.json.JSONOptions.<init>(JSONOptions.scala:109)\n\tat org.apache.spark.sql.catalyst.json.JSONOptionsInRead.<init>(JSONOptions.scala:133)\n\tat org.apache.spark.sql.catalyst.json.JSONOptionsInRead.<init>(JSONOptions.scala:139)\n\tat org.apache.spark.sql.execution.datasources.json.JsonFileFormat.inferSchema(JsonFileFormat.scala:55)\n\tat org.apache.spark.sql.execution.datasources.DataSource$$anonfun$6.apply(DataSource.scala:193)\n\tat org.apache.spark.sql.execution.datasources.DataSource$$anonfun$6.apply(DataSource."}
Thanks!
Ed