I need to implement an incremental load for sap tables via the sap table connector in azure data factory within a copy activity.
Currently, I am trying to set the watermark of the sap table (e.g. MAX(ERDAT)) through a lookup activity. Unfortunately, I cannot create a custom query, because the sap table connector is very limited and only supports basic filtering options (link).
Does someone know how to set a watermark via the sap table connector?
Thanks in advance!
https://docs.microsoft.com/en-us/azure/data-factory/load-sap-bw-data
This document providing the information regarding the watermark in SAP table connector. Check it out for the implementation.
Personally, I'd prefer loading the delta e.g."ERDAT = today" to an OpenHub, then loading that regularly to Azure.
If that option does not work, using the link you provided, you can create a custom function module on SAP, and filter out records based on date.
Use this link for more info https://www.cdata.com/kb/entries/sap-custom-read-table-function.rst
