Google Cloud Storage: 401 unauthorized error

user_12870513 0
Viewed 610

My target is to upload a file to GCP bucket from cloudhub and I am using service account and my only option is to use Json Key file.

This is what I've done:

  1. I have referred to this GCP information and this post to upload a file to GCP cloud storage and I am able to push the file via local but in Cloudhub I am not able to push the file to bucket.

I am receiving 401 unauthorized error.

In both the links a step is mentioned to set GOOGLE_APPLICATION_CREDENTIALS = "path-to-json-key" in environment variables.

When I try passing an absolute path(/Users/..json-key) locally it works and the file gets uploaded to bucket but when I try in cloudhub runtime manager passing ${app.home}/"json-key" or ${mule.home}/apps/${app.name}/"json-key" as value and it fails.

The file is located in src/main/resources directory and I am using java class (invoke static as mentioned in 2nd link) to upload the file. Need some guidance regarding this.

  1. An alternative approach I thought, is it possible somehow to use this json key file in Java class itself rather than passing it via environment variable?

  2. If at all the above two option does not work, any other approach wrt service account for app deployment in cloudhub ? (APIkey is an approach I tried but it does not restrict APIKey to a particular bucket)

Java Class

package com.mule.google.cloud.storage;

import com.google.cloud.storage.BlobId;
import com.google.cloud.storage.BlobInfo;
import com.google.cloud.storage.Storage;
import com.google.cloud.storage.StorageOptions;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Paths;
import static java.nio.charset.StandardCharsets.UTF_8;

public class Upload {
    public static void uploadObject(String projectId, String bucketName, String objectName, String filePath)
            throws IOException {

        Storage storage = StorageOptions.newBuilder().setProjectId(projectId).build().getService();
        BlobId blobId = BlobId.of(bucketName, objectName);
        BlobInfo blobInfo = BlobInfo.newBuilder(blobId).build();
        storage.create(blobInfo, filePath.getBytes(UTF_8));
    }
}

Error Log

Message               : Invocation of static Method 'uploadObject(java.lang.String,java.lang.String,java.lang.String,java.lang.String)' from Class 'com.mule.google.cloud.storage.Upload' with arguments [org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider arg0, org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider arg1, org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider arg2, org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider arg3] resulted in an error.
Expected arguments are [java.lang.String projectId, java.lang.String bucketName, java.lang.String objectName, java.lang.String filePath].
Cause: com.google.cloud.storage.StorageException - 401 Unauthorized
POST https://storage.googleapis.com/upload/storage/v1/b/"bucket-name"/o?projection=full&uploadType=multipart
Element               : upload-logFile/processors/3 @ poc-gcs-test:poc-mule-gcs.xml:305 (Invoke static)
Element DSL           : <java:invoke-static method="uploadObject(java.lang.String,java.lang.String,java.lang.String,java.lang.String)" doc:name="Invoke static" doc:id="3646e20f-0fef-4f57-84f8-d1ecc0f8afd5" class="com.mule.google.cloud.storage.Upload">
<java:args><![CDATA[
#[{
    arg0: vars.projectId,
    arg1: vars.bucketName,
    arg2: vars.objectName,
    arg3: vars.agg_msg default [],
    
}]
]]></java:args>
</java:invoke-static>
Error type            : JAVA:INVOCATION
FlowStack             : at upload-logFile(upload-logFile/processors/3 @ poc-gcs-test:poc-mule-gcs.xml:305 (Invoke static))
at consume-CustomerQ(consume-CustomerQ/processors/6 @ poc-gcs-test:poc-mule-gcs.xml:275 (Flow Reference upload-logFile))
Payload Type          : org.mule.runtime.core.internal.streaming.bytes.ManagedCursorStreamProvider
--------------------------------------------------------------------------------
Root Exception stack trace:
com.google.api.client.googleapis.json.GoogleJsonResponseException: 401 Unauthorized
POST https://storage.googleapis.com/upload/storage/v1/b/"bucket-name"/o?projection=full&uploadType=multipart

Note - The problem does not seem that the key file is incorrectly passed via environment. The java class or any package/dependency is not able to access it via environment with that relative path.

google-cloud-storage mule google-cloud-platform java
1 Answers

If the problem is that the file can not be located in CloudHub try instead this method to reference it as described in this KB article:

${mule.home}/apps/${app.name}/json-key

The method using the environment variable will not work on CloudHub because there is no way to set operating system environment variables by design. You need to add manual authentication code as described in GCP documentation. You can add a parameter to the Java method to receive above path calculated in the flow and use it to load the key in Java.

Example:

            var credential = GoogleCredential.FromFile(jsonPath);
            var storage = StorageClient.Create(credential);

Note that you should not be using System.out.println(). It doesn't scale and can cause issues. Use a Log4j2 logger to log from Java code.

user_7218550
Related