Move files from Regional to Multiregional Bucket in GCE

user_1820480 0
Viewed 262

I have a "nearline" + "europe-west4 (Netherlands)" bucket in the Google Computer Engine. If the files in the bucket reach a certain age (say, 6 months), I'd like to move them to a "coldline" + "mutli-regional (EU)" bucket. I tried implementing this behaviour using the bucket lifecycle rules, but unfortunately the documentation says, that you cannot move files from a "regional" to a "multiregional" bucket.

How can I implement this behaviour? Is there some other means of doing this apart from the bucket life cycle rules?

google-cloud-data-transfer google-compute-engine google-cloud-platform
1 Answers

Yes, accordingly to the documentation Lifecycle actions:

For buckets in a region, the new storage class cannot be Multi-Regional Storage.

As result, if you want to use Lifecycle to manage Storage Class of your data you should switch to using the same location region or multi-region. More information you can find in the documentation Bucket locations:

Use a region to help optimize latency and network bandwidth for data consumers, such as analytics pipelines, that are grouped in the same region.

and

Use a multi-region when you want to serve content to data consumers that are outside of the Google network and distributed across large geographic areas, or when you want the higher availability that comes with being geo-redundant.

As a possible workaround you can try Storage Transfer Service:

Storage Transfer Service allows you to quickly import online data into Cloud Storage. You can also set up a repeating schedule for transferring data, as well as transfer data within Cloud Storage, from one bucket to another.

You can schedule daily transfers between your region and multi-region buckets using as filter like Filter transferred files by last-modified time with Min age and Max age in hours, and options like Delete objects from source once they are transferred.

user_124287940
Related