Whenever I am pulling data from Salesforce object into Databricks, all the columns in the Databricks is showing as type String. But I want the same data type that is being used in Salesforce. Hence I am trying to use the inferSchema=true option while loading. But that is not working.
Below is the query I am using for loading Salesforce object:
val sfDF = spark.
read.
format("com.springml.spark.salesforce").
option("username", "your_salesforce_username").
option("password", "your_salesforce_password_with_secutiry_token"). //<salesforce login password><security token>
option("soql", soql).
option("inferSchema", "true").
option("version", "48.0").
load()
I am getting the below error message:
org.apache.spark.SparkException: Job aborted due to stage failure: Task 60 in stage 6231.0 failed 4 times, most recent failure: Lost task 60.3 in stage 6231.0 (TID 84032, 10.252.35.82, executor 99): java.lang.UnsupportedOperationException: Unexpected data type BooleanType
Is there any other way to inferSchema while pulling data from Salesforce object to Databricks? Any suggestion would be very helpfull.