I have a job that streams data from a delta table with parquet files to a an output table in json format. both tables live in an Azure Data Lake container.
I get the following error, which I can't make sense of:
java.lang.IllegalArgumentException: requirement failed: Did not get the first delta file version: 921 to compute Snapshot
What does this mean? I don't want to delete my checkpoint files or the transaction log etc.
Thanks in advance
Note: Restoring Azure Data Lake Storage Gen2 flat and hierarchical namespaces is not supported.
For more details, refer MSFT Document “Point-in-time restore”.
Point-in-time restore allows you to recover data from actions that only affected block blobs. Any activities that acted on containers are irreversibly lost. For example, if you use the Delete Container action to delete a container from the storage account, that container cannot be restored using a point-in-time restore operation. If you wish to restore individual blobs later, remove individual blobs rather than a whole container.
