How to throw Exception in Databricks?

user_8227399 0
Viewed 10899

I want my Databricks notebook to fail if a certain condition is satisfied. Right now I am using dbutils.notebook.exit() but it does not cause the notebook to fail and I will get mail like notebook run is successful. How can I make my notebook fail?

pyspark azure-databricks databricks apache-spark
3 Answers

Correct, although dbutils.notebook.exit("Custom message") makes the job skip rest of the commands, the job is marked as succeeded. We can use raise Exception if its a python notebook. This will also skip the rest of the commands, but mark the job as failed.

if condition: 
  raise Exception("Custom message")
user_86426080

For me the better way was to re-raise the same exception I got after handling... I've added some reporting I need in except: step, but then reraise, so job has status FAIL and logged exception in the last cell result.

Example:

try:
    run_my_job()
except Exception as err:
    log_to_some_place(f"{JOB_NAME} has failed, RunID: {mu_run_id}")
    raise
user_40344230

I am on the same boat and for sure dbutils.notebook.exit() is not helping in failing a job .There can be better way to get this done , but the below piece of code will make the job fail .

status = 'Fail'
if(status=='Fail'):
  10/0
else:
  100/10
user_111376790
Related