Google cloud pub/sub function gives "The requested snapshot version is too old" when querying firestore

Viewed 3787

I have a gcloud pub/sub-function that performs a simple query on a collection. It was working fine before Oct 08. Now I am seeing "The requested snapshot version is too old" error messages.

I have created an HTTP function with the same code and run it manually, it works perfectly fine.

Here is the function:

// 0 3 * * * - at 03:00 AM every day
exports.GenerateRankings = functions.pubsub.schedule('0 3 * * *')
    .onRun((context) => {

        console.log("GenerateRankings Task started")

        const playersCollection = admin.firestore().collection('players')

        playersCollection.orderBy("Coin", "desc").get()
            .then((qs) => {
                console.log("Fetching Players by Coin")
                // some staff
                return true
            })
            .catch((error) => {
                console.error("Error fetching players", error)
                return false
            })
    })

And here is the error stack:

9 FAILED_PRECONDITION: The requested snapshot version is too old.
 at Object.callErrorFromStatus (/workspace/node_modules/@grpc/grpc-js/build/src/call.js:31:26)
 at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client.js:327:49)
 at Object.onReceiveStatus (/workspace/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:305:181)
 at /workspace/node_modules/@grpc/grpc-js/build/src/call-stream.js:124:78
 at processTicksAndRejections (internal/process/task_queues.js:79:11)
 Caused by: Error
 at Query._get (/workspace/node_modules/@google-cloud/firestore/build/src/reference.js:1466:23)
 at Query.get (/workspace/node_modules/@google-cloud/firestore/build/src/reference.js:1455:21)
 at /workspace/index.js:22:47
 at cloudFunction (/workspace/node_modules/firebase-functions/lib/cloud-functions.js:130:23)
 at /layers/google.nodejs.functions-framework/functions-framework/node_modules/@google-cloud/functions-framework/build/src/invoker.js:198:28
 at processTicksAndRejections (internal/process/task_queues.js:97:5) {
    code: 9,
    details: 'The requested snapshot version is too old.',
    metadata: Metadata { internalRepr: Map {}, options: {} }
 }

I know there is another unanswered question "The requested snapshot version is too old." error in Firestore similar to this. I am facing this problem with pub/sub-functions.

Thanks for your help.

4 Answers

In case someone faces this problem, I answer my own question.

After a lot of reading, testing, I've noticed there is a warning in my logs like "Function returned undefined, expected Promise or value". Because I return a promise in my function, I was not paying attention to this.

Adding a return on top of my function fixed my warning, and my function has been running successfully for 5 days.

return exports.GenerateRankings = functions.pubsub.schedule('0 3 * * *') ...

I encountered the same problem, in my case, the code was as follows and the error mentioned in this issue was returned.

const userSnapshots = await this.firestore.collection('Users').where('archive', '==', false).get();
const users = userSnapshots.docs
users.map(async (user) => {
    //code to update user documents
});

Then I changed the code to the following and it worked without returning the error mentioned in this issue.

const userSnapshots = await this.firestore.collection('Users').where('archive', '==', false).get();
const users = userSnapshots.docs
const markPromises = users.map(async (user) => {
    //code to update user documents
});

await Promise.all(markPromises);

I don’t know this is the correct answer to this question but this worked for me.

For my case, I guess the problem was because of the size of data, and time it took to do the data management. I have split data into parts of 5000 elements :

    const store = getFirestore(initializeApp(myConfig));
    const collectionRef = store.collection("users");
    const limit = 5000;
    let docs: Array<QueryDocumentSnapshot<DocumentData>>;
    let querySnapshot: QuerySnapshot<DocumentData>;

    let query = collectionRef.limit(limit);

    do {
        querySnapshot = await query.get();
        docs = querySnapshot.docs; // thanks to @prahack answer
        for (const doc of docs) {
            await manageMyData(doc.id, doc.data());
        }
        if (docs.length > 0) {
            // Get the last visible document
            query = collectionRef.startAfter(docs[docs.length - 1]).limit(limit);
        }
    } while (docs.length > 0);

Without my data managing, i can use part of 10.000 elements (limit = 10000), but with management, i should down to 5000.

I spent hours trying to run a list of actions inside a "pubsub" and then I decided to try the following code and it worked.

this.firestore.collection('Users').where('archive', '==', false).get().then(snap=>{
    // Code Here
});
Related