What is the best practice to delete multiple objects(files and folders) from google cloud storage using node.js

user_1785053 0
Viewed 529

I want to delete multiple objects from my google cloud storage, which means delete folders and files in combination. There is an example in the official documentation:

bucket.deleteFiles({ prefix: 'image-' }, callback);

This will delete all files in the images directory. But what if I want to delete 10 000 files in this directory? My demo code is this:

const _deleteItems = async (gcsClient, bucketName, allItems) => {
    try {
        for (let item of allItems) {
            await gcsClient.bucket(bucketName).deleteFiles({ prefix: getFilePath(item) }, function(err) {});
        }
    } catch (err) {
        throw new Error('Could not delete item');
    }
};

Is there a better way to do this? What if someone else in the meantime upload new file in the folder that i am going to delete? That file also will be deleted in this case.

google-cloud-storage node.js google-cloud-platform
1 Answers

  • Is there a better way to do this?

    Here you may find the best practices regarding object deletion and bulk object deletion which applies to your use case.

  • What if someone else in the meantime upload new file in the folder that i am going to delete? That file also will be deleted in this case.

    In this case you need to use Object Lifecycle Management and set a rule to protect files with a certain "age". You can also use Object versioning which "protects your Cloud Storage data from being overwritten or accidentally deleted" but please keep in mind the following:

    • "Enabling Object Versioning increases storage costs"

    • "When using Object Lifecycle Management in buckets with object versioning enabled, deleting the live version of an object creates a noncurrent version, while deleting a noncurrent version deletes that version permanently."

user_119281300
Related