Error: ENOENT: no such file or directory even when file exists in Firebase Cloud Functions

Viewed 3669

I'm relatively new to Cloud Functions and have been trying to solve this issue for a while. Essentially, the function I'm trying to write is called whenever there is a complete upload onto Firebase Cloud Storage. However, when the function runs, half the time, it runs to the following error:

The following error occured:  { Error: ENOENT: no such file or directory, open '/tmp/dataprocessing/thisisthefilethatiswritten.zip'
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/tmp/dataprocessing/thisisthefilethatiswritten.zip' }

Here's the code:

const functions = require('firebase-functions');
const admin = require('firebase-admin')
const inspect = require('util').inspect
const path = require('path');
const os = require('os');
const fs = require('fs-extra');

const firestore = admin.firestore()
const storage = admin.storage()

const runtimeOpts = {
    timeoutSeconds: 540,
    memory: '2GB'
  }

const uploadprocessing = functions.runWith(runtimeOpts).storage.object().onFinalize(async (object) => {
    const filePath = object.name
    const fileBucket = object.bucket
    const bucket_fileName = path.basename(filePath);
    const uid = bucket_fileName.match('.+?(?=_)')
    const original_filename = bucket_fileName.split('_').pop()

    const bucket = storage.bucket(fileBucket);
    const workingDir = path.join(os.tmpdir(), 'dataprocessing/');
    const tempFilePath = path.join(workingDir, original_filename);

    await fs.ensureDir(workingDir)

    await bucket.file(filePath).download({destination: tempFilePath})

    //this particular code block I included because I was worried that the file wasn't
    //being uploaded to the tmp directly, but the results of the function 
    //seems to confirm to me that the file does exist.

    await fs.ensureFile(tempFilePath)
        console.log('success!')
        fs.readdirSync(workingDir).forEach(file => {
            console.log('file found: ', file);
            });
        console.log('post success')
        fs.readdirSync('/tmp/dataprocessing').forEach(file => {
            console.log('tmp file found: ', file);
    })


    fs.readFile(tempFilePath, function (err, buffer) {
        if (!err) {
            //data processing comes here. Please note that half the time it never gets into this
            //loop as instead it goes into the else function below and outputs that error.
        }
        else {
            console.log("The following error occured: ", err);
        }
    })
    fs.unlinkSync(tempFilePath);
    return
})
module.exports = uploadprocessing;

I've been trying so many different things and the weird thing is that when I add code into the "if (!err)" (which doesn't actually run because of the err) it just arbitrarily starts working sometimes quite consistently, but then it stops working when I add different code. I would have assumed that the issue arises from the code that I added, but then the error comes up literally when I just change/add/remove comments as well... Which should technically have no effect on the function running...

Any thoughts? Thank you in advance!!! :)

1 Answers

fs.readFile is asynchronous and returns immediately. Your callback function is invoked some time later with the contents of the buffer. This means that fs.unlinkSync is going to delete the file at the same time it's being read. This means you effectively have a race condition, and it's possible that the file will be removed before it's ever read.

Your code should wait until the read is complete before moving on to the delete. Perhaps you want to use fs.readFileSync instead.

Related