Cannot import Cosmosdb in databricks

user_3254986 0
Viewed 267

I setup a new cluster on databricks which using databricks runtime version 10.1 (includes Apache Spark 3.2.0, Scala 2.12). I also installed azure_cosmos_spark_3_2_2_12_4_6_2.jar in Libraries.

I create a new notebook with Scala

import com.microsoft.azure.cosmosdb.spark.schema._
import com.microsoft.azure.cosmosdb.spark.CosmosDBSpark
import com.microsoft.azure.cosmosdb.spark.config.Config

But I still get error: object cosmosdb is not a member of package com.microsoft.azure

Does anyone know which step I missing? Thanks

azure-cosmosdb azure-databricks scala
2 Answers

Looks like the imports you are doing are for the older Spark Connector (https://github.com/Azure/azure-cosmosdb-spark).

For the Spark 3.2 Connector, you might want to follow the quickstart guides: https://docs.microsoft.com/azure/cosmos-db/sql/create-sql-api-spark

The official repository is: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/cosmos/azure-cosmos-spark_3-2_2-12

Complete Scala sample: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/cosmos/azure-cosmos-spark_3_2-12/Samples/Scala-Sample.scala

Here is the configuration reference: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/cosmos/azure-cosmos-spark_3_2-12/docs/configuration-reference.md

user_56415980

You may be missing the pip install step:

pip install azure-cosmos

user_192771280
Related