How to set encoding to UTF16 in PySpark Oracle JDBC Connection

user_16935612 0
Viewed 276

I'm trying to load the data from Oracle to Databricks but I encountered a Unicode character issue in PySpark it can't encode the Unicode character as per the format available in Oracle in fact it displays the replacement character as '▯'.In oracle the NLS_NCHAR_NCHARACTERSET=AL16UTF16.

I tried the Inserting national characters into an oracle NCHAR or NVARCHAR column does not work Oracle JDBC system property but it doesn't work in my case. I request you to please provide an alternative to fix this issue.

pyspark databricks jdbc unicode oracle
1 Answers

option 1 - Use a jdbc option when you read data with spark:

spark.read.format("jdbc")...option("useUnicode", true).option("characterEncoding", "UTF-16")

option 2 - Use proper connection string:

url = "...?useUnicode=true&characterEncoding=UTF-16"
spark.read.format("jdbc").option("url", url)
user_12243890
Related