I tried to debug in spark-shell with Apache IoTDB, and find that error occured and stated that here is a conflict of thrift version between IoTDB and Spark.
Can I still execute my operation in spark-shell?
1 Answers
if you want to debug in spark-shell, you need to execute rm -f $SPARK_HOME/jars/libthrift* and cp $IOTDB_HOME/lib/libthrift* $SPARK_HOME/jars/ to resolve it. Otherwise, you can only debug the code in IDE. If you want to run your task by spark-submit, you must package with dependency.
Related