Transfer huge data from maria db to ssms when their is data change in mariadb

Question

I did tried using schedular for Databricks notebook but it is creating unnecessary of loading data. The data in mariadb changes randomly it is not fixed , if I try pipeline I cant call a trigger for change in data and transfer of data from one Database to another.

Please help me with any pipeline ideas , azure datafactory ideas or python codes as well so that I can transfer tables when their are changes in Mariadb

user_11104805 · Answer

One way to trigger pipeline is to use event-based trigger.

Creating event-based trigger in Azure Data Factory. Create trigger and select Type as Custom events.

Refer - https://www.mssqltips.com/sqlservertip/6063/create-event-based-trigger-in-azure-data-factory/

Second way is to use logic app. This is the best approach for your query.

Refer this answer by Trent Tamura

Transfer huge data from maria db to ssms when their is data change in mariadb

1 Answers