Run a gsutil command in a Google Cloud Function

user_4262057 0
Viewed 2794

I would like to run a gsutil command every x minutes as a cloud function. I tried the following:

# main.py
import os

def sync():
     line = "gsutil -m rsync -r gs://some_bucket/folder gs://other_bucket/other_folder"
     os.system(line)

While the Cloud Function gets triggered, the execution of the line does not work (or i.e. the files are not copied from one bucket to another). However, it does work fine when I run it locally in Pycharm or with cmd. What is the difference with cloud functions?

gsutil google-cloud-functions google-cloud-storage google-cloud-platform python
3 Answers

You can use Cloud Run for this. You have very few change to perform in your code.

Create a container with gsutil installed and python also, for example gcr.io/google.com/cloudsdktool/cloud-sdk as base image

Take care of the service account used when you deploy Cloud Run, grant the correct permission for accessing to your bucket

Let me know if you need more guidance

user_113725930

Cloud Functions server instances don't have gsutil installed. It works on your local machine because you do have it installed and configured there.

I suggest trying to find a way to do what you want with the Cloud Storage SDK for python. Or figure out how to deploy gsutil with your function and figure out how to configure and invoke it from your code, but that might be very difficult.

user_8071260

There's no straightforward option for that.

I think the best for Cloud Functions is to use google-cloud-storage python library

user_11191530
Related