PySpark: How can I import a GitHub library into Databricks?

You can clone the repo and create a Python package as explained here : https://github.com/titicaca/spark-iforest:

Step 2. Package pyspark-iforest and install it via pip, skip this step if you don't need the python pkg

cd spark-iforest/python

python setup.py sdist

pip install dist/pyspark-iforest-<version>.tar.gz

Here you only need the 2 first commands to generate the package but you have to change the second one to generate an egg package instead of source distribution package:

python3 setup.py bdist_egg

Now, you'll find the file in /dist folder:

pyspark_iforest-2.4.0-py3.7.egg

Finally, on Databricks, select Create > Library and choose Python Egg to upload the generated file. More details can be found here.

You can also use %pip to install notebook-scoped libraries (see documentation) from GitHub.

If you have a location mounted on Databricks you can also copy the pyspark-iforest-<version>.tar.gz file there and pip install from there. Or use the upload functionality if it's enabled, see here.

%sh git clone https://github.com/titicaca/spark-iforest

Then make sure it is cloned in the databricks driver root path

%sh ls -al 

And then pip install if desired using:

%pip install ./{yourpackage_name}

If it returns message that the requirement is already satisfied, you have to do:

 %pip uninstall -y {yourpackage_name} 

This would remove any version of the libraries that may have installed from PyPi instead of the git clone version. Next, redo the pip install package in databricks.