Azure Purview - Deleted folders in ADLS2

user_2061133 0
Viewed 488

We are scanning ADLS Gen 2 data lake successfully with Purview. However, if a folder is deleted in the lake and you re-scan, the scan does not remove the deleted folder. The deleted folder remains in Purview, but the last modified date (from the scan) remains as the previous scan date/time from when it was present. How can I purge these now invalid entries? Removing the previous scan does not work. Removing the entire source from Purview leaves the scan results behind in the register and a new scan does not clean them up. There is also no manual delete/purge option. The only option seems to be to remove the entire purview account from Azure, redeploy and reconfigure everything.

Am I missing a trick?

azure-purview azure-data-lake azure
1 Answers

Reading to this https://docs.microsoft.com/en-us/azure/purview/concept-detect-deleted-assets this mostly seems like a expected behavior, did you try scanning more than twice post 5mins intervals.

To keep deleted files out of your catalog, it's important to run regular scans. The scan interval is important, because the catalog can't detect deleted assets until another scan is run. So, if you run scans once a month on a particular store, the catalog can't detect any deleted data assets in that store until you run the next scan a month later.

user_159694170
Related