Working with google cloud storage in julia applications

user_14847557 0
Viewed 259

I have a query related to the google cloud storage for julia application. Currently, I am hosting a julia application (docker container) on GCP and would like to allow the app to utilize cloud storage buckets to write and read the data.

I have explored few packages which promise to do this operation.

  1. GoogleCloud.jl This package in the docs show a clear and concise representation of the implementation. However, adding this package result in incremental compilation warning with many of the packages failing to compile. I have opened an issue on their github page : https://github.com/JuliaCloud/GoogleCloud.jl/issues/41

  2. GCP.jl The scope is limited, currently the only support is for BigQuery

  3. Python package google This is quite informative and operational but will take a toll on the code's performance. But do advise if this is the only viable option.

I would like to know are there other methods which can be used to configure a julia app to work with google storage?

Thanks look forward to the suggestions!

google-cloud-storage julia google-cloud-platform
1 Answers

GCP.jl is promising plus you may be able to do with gRPC if Julia support gRPC (see below).

Discovery

Google has 2 types of SDK (aka Client Library). API Client Libraries are available for all Google's APIs|services.

Cloud Client Libraries are newer, more language idiosyncratic but only available for Cloud. Google Cloud Storage (GCS) is part of Cloud but, in this case, I think an API Client Library is worth pursuing...

Google's API (!) Client Libraries are auto-generated from a so-called Discovery document. Interestingly, GCP.jl specifically describes using Discovery to generate the BigQuery SDK and mentions that you can use the same mechanism for any other API Client Library (i.e. GCS).

NOTE Explanation of Google Discovery

I'm unfamiliar with Julia but, if you can understand enough of that repo to confirm that it's using the Discovery document to generate APIs and, if you can work out how to reconfigure it for GCS, this approach would provide you with a 100% fidelity SDK for Cloud Storage (and any other Google API|service).

Someone else tried to use the code to generate an SDK for Sheets and had an issue so it may not be perfect.

gRPC

Google publishes for the subset of its services that support gRPC. If you'd prefer to use gRPC, it ought be possible to use the Protobufs in Google's repo to define a gRPC client for Cloud Storage

user_6092900
Related