spark.read gives KrbException in Databricks

Question

I am trying to connect to an sql database from the databricks notebook. The following is my code:

jdbcDF = spark.read \
    .format("com.microsoft.sqlserver.jdbc.spark") \
    .option("url", "jdbc:sqlserver://dbs-sdp-sql-dev.database.windows.net:1433;database=dm_test;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;Authentication=ActiveDirectoryIntegrated") \
    .option("dbtable", "Cars") \
    .option("Driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver").load()

I receive the error:

IllegalArgumentException: KrbException: Cannot locate default realm

The full trace is:

IllegalArgumentException                  Traceback (most recent call last)
 in 
----> 1 jdbcDF = spark.read \
      2         .format("com.microsoft.sqlserver.jdbc.spark") \
      3         .option("url", "jdbc:sqlserver://dbs-sdp-sql-dev.database.windows.net:1433;database=dm_test;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;Authentication=ActiveDirectoryIntegrated") \
      4         .option("dbtable", "Cars") \
      5         .option("Driver", "com.microsoft.sqlserver.jdbc.SQLServerDriver").load()

/databricks/spark/python/pyspark/sql/readwriter.py in load(self, path, format, schema, **options)
    208             return self._df(self._jreader.load(self._spark._sc._jvm.PythonUtils.toSeq(path)))
    209         else:
--> 210             return self._df(self._jreader.load())
    211 
    212     def json(self, path, schema=None, primitivesAsString=None, prefersDecimal=None,

/databricks/spark/python/lib/py4j-0.10.9-src.zip/py4j/java_gateway.py in __call__(self, *args)
   1302 
   1303         answer = self.gateway_client.send_command(command)
-> 1304         return_value = get_return_value(
   1305             answer, self.gateway_client, self.target_id, self.name)
   1306 

/databricks/spark/python/pyspark/sql/utils.py in deco(*a, **kw)
    121                 # Hide where the exception came from that shows a non-Pythonic
    122                 # JVM exception message.
--> 123                 raise converted from None
    124             else:
    125                 raise

IllegalArgumentException: KrbException: Cannot locate default realm

I have tried installing python-krb5 on the cluster but it fails.

I have tried installing requests-kerberos==0.12.0 on the cluster but it fails.

I am following a tutorial at: https://docs.microsoft.com/en-us/sql/connect/spark/connector?view=sql-server-ver15

How do I fix this error?

spark.read gives KrbException in Databricks

0 Answers