How to install a jar in databricks using ADF

user_9298966 0
Viewed 595

We are able to install the jar file using the UI method to a particular cluster. But our requirement to install it on all the ondemand clusters in the workspace. We are using the below shell script to download the jar file to DBFS. Not sure how we can refer/install this jar in all cluster using a global init script

curl https://repo1.maven.org/maven2/com/databricks/spark-xml_2.12/0.12.0/spark-xml_2.12-0.12.0.jar >/dbfs/FileStore/jars/maven/com/databricks/spark_xml_2_12_0_12_0.jar

Any help would be really appreciated!!

azure-databricks databricks bash
2 Answers

There is an alternate solution for adding jar library to the job cluster which is called from Azure data factory while running our job.

In ADF, while calling the notebook we have the option to include the jar directory in DBFS or we can able to give the Maven coordinates.

ADF SETTINGS

user_88568090

In the global init script you can just download this file into /databricks/jars/ directory - then it will be picked up by cluster

Alex_Ott_186270
Related