Can I ingest data into tables in azure data explorer through Databricks?

user_18215816 0
Viewed 58

I have to ingest data from ADLS gen2 to tables in ADX. I first tried to use ADF, but considering the run time and everything, this was an really inefficient way to do it. I think it would be so much easier if there is a way to ingest data into ADX via databricks. I only can access ADLS through Service Pricipals so I can't directly connect to the storage account from ADX.

Any help will be much appreciated!

azure-data-explorer azure-databricks azure
2 Answers

If it is for continuous ingestion consider creating an Event Grid data connection

If it is for one-time ingestion consider using the "lightIngest" tool, and provide the applicable connection string for your use case.

user_101679140

I would also recommend the Event Grid Data connection. Please note that you can set up this data connection using managed identities.

user_14278260
Related