Is there a way to recover deleted data in Azure Databricks?

Viewed 2973

With out realizing shift+enter runs a cell. I was writing a delete from table and pressed shift enter which deleted all of the data in table.

2 Answers

In a Delta Lake table, the DELETE is another transaction, the data is only 'marked for deletion' not immediately deleted. Using the Time Travel feature, you can view your transaction history and then select from the version prior to the SQL DELETE and insert into the same table to restore your data.

To restore the data:

DESCRIBE HISTORY <table>

Note down version number prior to the delete

INSERT INTO <table> SELECT * from <table> VERSION AS OF <version from history>


Let me add, that as of DBR 7.4, the RESTORE command is available:

RESTORE [TABLE] table_identifier[TO] <time_travel_version>

Per Azure Databricks Delta Lake Docs

It’s down to where the data was stored. If in the default DBFS location then it’s gone I’m afraid. That uses a blob account with no backup features.

If you mounted your own blob/lake storage and enabled soft delete or snapshots you can get it back by going to that resource in the azure portal.

If it’s a relational database source then you may have backups.

But chances are it is gone I’m afraid.

Related