How can we resolve DB2 java.io.CharConversionException during Spark submission

user_2038576 0
Viewed 2547

I am getting this below error during spark submit:

org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala) Caused by: com.ibm.db2.jcc.am.SqlException: [jcc][t4][1065][12306][4.17.28] Caught java.io.CharConversionException. See attached Throwable for details. ERRORCODE=-4220, SQLSTATE=null at com.ibm.db2.jcc.am.fd.a(fd.java:731)

As per IBM documentation Click here, I added this below property but till getting same error. I am using db2jcc4.jar driver.

Is there any way to pass db2.jcc.charsetDecoderEncoder during spark submission.

System.setProperty("db2.jcc.charsetDecoderEncoder", "3")    
val prop = new Properties()
prop.put("driver", driver)
prop.put("db2.jcc.charsetDecoderEncoder", "3")
prop.put("-Ddb2.jcc.charsetDecoderEncoder", "3")
prop.put("continueBatchOnError", "true")`

sqlContext.read.jdbc(
  url = url,
  table = table,
  predicates = parts,
  connectionProperties = prop)
apache-spark db2 scala
1 Answers

Finally I have resolved DB2 java.io.CharConversionException issue only passing two parameter to spark driver & executor. 

--conf 'spark.executor.extraJavaOptions=-Ddb2.jcc.charsetDecoderEncoder=3' 
--conf 'spark.driver.extraJavaOptions=-Ddb2.jcc.charsetDecoderEncoder=3'
user_20385760
Related