Import/Export (Azure) Databricks Notebooks programatically

user_6346825 0
Viewed 877

I have a databricks notebook that takes as an input the location of the table and then generates graphs.

I can run this notebook from a wrapper notebook for many different tables.

Is it possible that every time that the notebook run, I save it with the results as an html in the databricks files system.

In essence, I want to programmatically export the notebook, in the same way as I would manually do File>Export>HTML

Is that possible? if yes, how?

Note: I was thinking that ,if there is nothing out of the box, I guess that the notebooks will be saved somewhere internally in the driver. I could get it from there and move it where I want with dbutils.

azure-databricks databricks
1 Answers

IN general you can export notebook using either REST API, via the export endpoint of workspace API - you can specify that you want to export as HTML. Another option is to use workspace export command of the Databricks CLI that uses REST API under the hood, but it's easier to use.

But in your case, the notebook (most probably, if you use dbutils.notebook.run) is executed as a separate job, so you need to use Runs Export API instead.

To call the API you need to have personal access token and host name, but it's easy to retrieve it programmatically from inside of the notebook. See this answer for exact details.

P.S. Notebook is not an object on file system, or something like - it exists only in memory, and not available on driver node. Maybe it will change with the upcoming Repos feature.

Alex_Ott_186270
Related