I have an algorithm that is written in C.
I want to use Spark to run it on a cluster of nodes. How can I run my C executable on Spark? I have read about JNI and Java Process to execute my binary.
1 Answers
Here is a nice article from databricks on how we can run c and c++ codes in apache spark.
https://kb.databricks.com/_static/notebooks/scala/run-c-plus-plus-scala.html
After moving the c/c++ codes to the underlying distributed file system, You can compile the code and run on spark worker nodes.
Related