Connecting Springboot application to Azure databricks

user_8855565 0
Viewed 508

I'm trying to connect SpringBoot Application to Azure Databricks. Below is something I have tried....

application.properties

spring.datasource.url = jdbc:spark://adb-**********.*.azuredatabricks.net:**/default;transportMode=http;ssl=1;httpPath=sql/protocolv1/o/******/******-*****-abcd341
spring.datasource.username = username
spring.datasource.password = Generated Token

pom.xml

Below are some dependencies I'm using...

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-core_2.10</artifactId>
    <version>1.5.2</version>
</dependency>

<dependency>
    <groupId>org.apache.spark</groupId>
    <artifactId>spark-sql_2.10</artifactId>
    <version>1.5.2</version>
</dependency>

<dependency>
    <groupId>com.databricks</groupId>
    <artifactId>spark-avro_2.10</artifactId>
    <version>2.0.1</version>
</dependency>

I'm getting below error..

***************************
APPLICATION FAILED TO START
***************************

Description:

Cannot determine embedded database driver class for database type NONE

Action:

If you want an embedded database please put a supported one on the classpath. If you have database settings to be loaded from a particular profile you may need to active it (no profiles are currently active).

Suggest me if I'm missing any maven dependency.

Thanks in Advance..

spring-boot azure-databricks databricks apache-spark maven
2 Answers

To connect from the Spring Boot you need to use JDBC driver, not Spark jars (remove them - you don't need them). You can get JDBC driver as described in documentation, or very recently - directly via Maven using following coordinates:

    <dependency>
      <groupId>com.databricks</groupId>
      <artifactId>databricks-jdbc</artifactId>
      <version>2.6.25-1</version>
    </dependency>

and then use standard JDBC APIs exposed by Spring. I have a simple example that uses JdbcTemplate to access data in Databricks - you just need to construct JDBC URL correctly:

String host = "";
String httpPath = "";
String token = "";

String jdbcUrl = "jdbc:databricks://" + host + 
       ":443/default;transportMode=http;ssl=1;httpPath=" +
       httpPath + ";AuthMech=3;UID=token;PWD=" + token;

and then just access data:

// define data source
SimpleDriverDataSource ds = new SimpleDriverDataSource();
ds.setDriver(new Driver());
ds.setUrl(jdbcUrl);
JdbcTemplate jdbcTemplate = new JdbcTemplate(ds);

// query data
List<Map<String, Object>> data = jdbcTemplate.queryForList(query);
for (Map<String, Object> row: data) {
....
}

P.S. You may omit username or at least set it to the token value...

Alex_Ott_186270

Try adding the "spring.datasource.driverClassName" , and let me know if that helps you to proceed

user_110073720
Related