Google Storage - Backup Bucket with Different Key For Each File

user_4869599 0
Viewed 27

I need to backup a bucket in which every file is encrypted with a different key to a different bucket on Google Storage.

I want to create a daily snapshot of the data so in a case where the data has been deleted I could easily recover it.

My Research:

  1. Using gsutil cp -r - because every file has a different key it does not work
  2. Using Google Transfer | cloud - does not work on such buckets from the same reason
  3. List all the files in the bucket and fetch all the keys from the database and copy each file - this will probably be very expensive to do because i have a lot of files and i want to do it daily
  4. Object versioning - Does not cover a case where the bucket has been completely deleted

Are there any other solutions for that problem?

google-cloud-storage
1 Answers

Unfortunately, as you mentioned the only option indeed, would be to follow your number 3 choice. As you said and as clarified in this official documentation here, download of encrypted data is a restricted feature, so you won't be able to download/snapshot the data, without fetching the keys and then copying the files.

Indeed, this will probably make a huge impact in your quota and pricing, since you will be performing multiple operations everyday, for multiple files, which will affect multiple aspects on the pricing. However, this seems to be the only available way right now. In addition to this, I would recommend you to raise a Feature Request in Google's Issue Tracker, so they can check about the possibility of implementing this in the future.

Let me know if the clarifed your doubts!

user_127672570
Related