Databricks Job fails with 'IPythonShell' object has no attribute 'ask_exit'

user_17601058 0
Viewed 331

I'm bringing some data from HTTP api and I've used exit() after printing the response but not used anywhere that ask_exit() still the job is throwing the error 'IPythonShell' object has no attribute 'ask_exit'. But when I run the same code manually it is getting executed successfully. The below is the code where it is failing.

      if appflds_httpResponse.status_code != 200:
            print("HTTP Response Status: ", appflds_httpResponse.status_code)
            exit()

I'm using the below:

import requests
import pandas as pd
import pandas as pd
import json
import xml.etree.ElementTree as ET
import lxml
from lxml import etree
import html
import sys
from requests import status_codes
import re
from pyspark.sql import *

could you kindly help on this how to resolve the issue.

thank you Amar

databricks python-3.x
2 Answers

instead of using exit(), the recommended way is to raise an exception - this will terminate job with correct exit code. Like this:

if appflds_httpResponse.status_code != 200:
  raise Exception(f"Non-200 HTTP status code: {appflds_httpResponse.status_code}")
Alex_Ott_186270

Thanks for the reply. The issue got resolved. But how?

We are migrating from the current DB to new DB. there was a setup already running as part of the current DB data loading which is fetching using the D.Bricks notebook1. the issue occurred due to the same dbricks notebook1 has started running by the other DB Job and hence there was a conflict of the target in which the job has to run. The new one has removed for the time being to run the same D.bricks notebook and resolved that conflict. Thank you Amarnath

user_176010580
Related