Optimize firebase realtime database search

Question

Below is my table structure in firebase realtime database. I have an array of ids and want to check if those ids exists in any of these refs data/userId/tests1 or data/userId/tests2. If the id don't exist in both if these ref I am adding it to my resultFilteredUsers array. Below is the solution I currently have but this is very slow. Is there any faster approach than this?

data
    userId
        tests1
            id1
            id2
        tests2
            id3
            id4

code

async function filter(userId: String, testIdsArray:string[]) {
    
        var filteredUsersSet1: string[] = []
        var resultFilteredUsers: string[] = []
    
        var db = FirebaseAdmin.database()
        var ref1 = db.ref("data/" + userId + "/tests1/")
        var ref2 = db.ref("data/" + userId + "/tests2/")
    
        await Promise.all(
            testIdsArray.map(id => {
                return ref1.child(id).once('value')
                  .then(snapshot => {
                    if (!snapshot.exists()){
                        filteredUsersSet1.push(id)
                    }
                    return snapshot;
                  }).catch((error: any) => {
                      logger.error("Error");
                    });
                })
            )
    
        await Promise.all(
            filteredUsersSet1.map(id => {
                return ref2.child(id).once('value')
                .then(snapshot => {
                    if (!snapshot.exists()){
                        resultFilteredUsers.push(id)
                    }
                    return snapshot;
                  }).catch((error: any) => {
                      logger.error("Error");
                    });
                })
            )
        return resultFilteredUsers;
    }

Frank van Puffelen · Answer

Since you're using await for the two Promise.all calls, you are ensuring that they happen sequentially. I don't think your use-case requires that in any way, so it'd be faster to combine all those promises in a single Promise.all call:

const promises1 = testIdsArray.map(id => {
    return ref1.child(id).once('value')
      .then(snapshot => {
        if (!snapshot.exists()){
            filteredUsersSet1.push(id)
        }
        return snapshot;
      }).catch((error: any) => {
          logger.error("Error");
        });
    })
)
    
const promises2 = filteredUsersSet1.map(id => {
    return ref2.child(id).once('value')
    .then(snapshot => {
        if (!snapshot.exists()){
            resultFilteredUsers.push(id)
        }
        return snapshot;
      }).catch((error: any) => {
          logger.error("Error");
        });
    })
)

await Promise.all(promises1.concat(promises2));

The above is more efficient, but won't be orders of magnitude faster.

Optimize firebase realtime database search

1 Answers