Why is google cloud storage not allowing file upload when public access is granted?

user_13222359 0
Viewed 627

In my node.js server I have a post request making 2 uploads to google cloud storage--each into a different bucket.

When I tested the functionality, I was able to successfully upload 2 files to 2 different buckets, but upon the next test, 1 of the 2 uploads is failing and throwing an error: Error: Could not load the default credentials.

Why would it fail on the second test on only 1 of the uploads?
Why would it say the credentials can't be loaded if it's a fully public bucket (all users have full object access to read/write/admin)?

app.post("/upload", upload.single("image"), (req, res) => {
    //this takes an image file and uploads it
    async function uploadFile() {
        await storage.bucket(bucketName).upload(imagePath, {destination: imagePath})
     }

    uploadFile().catch(console.error);

    //this resizes the image 
    Jimp.read(imagePath, (err, img) => {
        if (err) throw err
        img.resize(Jimp.AUTO, 300).write(imgTitle + "-thumb.jpg")
    })

    //this uploads the resized image
    async function uploadThumb() {
      await storage.bucket(thumbs).upload(imgTitle + "-thumb.jpg", {destination: cat + "/" + subCat + "/" + imgTitle + "-thumb.jpg"})
    }
    setTimeout(() => {    //this timeout waits 2 seconds for JIMP to finish processing the image
        uploadThumb().catch(console.error);
    }, 2000)
});

I'm hoping someone can explain why this stopped working after the first test. The function that uploads the resized image works in both tests, but the function that uploads the original file fails on the 2nd test throwing the error: Error: Could not load the default credentials

UPDATE

After many tests, I have possibly deduced that this is a file size issue. The thumbnail upload works every time, while the full size image fails when its size reaches ~2-3MB. Reading the GCS docs, it says that 5TB is the maximum single file upload limit, so I don't know why there is an issue with a few MB. I do not want to lower the image size/resolution as they are art works that will need to be viewed at full size (that's exactly why I'm creating the thumbnails in the first place).

google-cloud-storage node.js file-upload
2 Answers

This issue was resolved by me through thorough research and trial and error (weeks, I'm not proud). I knew I needed to add verification parameters into my const storage = new Storage(); (not in my question, and I know now it should have been).

Originally I was tyring to use a .env file to pass in the project-id and client_secrets.json file in every way I knew how. But after rereading the documentation for the nth time, I came across the correct syntax for it.

I needed to create constants of the project-id and the location of the .json file it resides in and pass those in as parameters like this:

const projectId = 'project-id';
const keyFileName = "client_secret.json";
const storage = new Storage({projectId, keyFileName});

The reason the uploads were failing some of the time was because I was getting some sort of a free pass to upload smaller objects, but as soon as the uploaded file size reach ~3 mb it would require verification that was not present. I still don't fully understand it, but this is how I solved it.

user_132223590

It looks like you're credentials are not loaded. Service credentials that is. Have you already tried running?

gcloud auth application-default login

Also, are the bucket permission identical for both?

user_86687180
Related