"Wrong app reference" when loading documents from Firestore with Python

user_14841175 0
Viewed 115

I have a database in a GCP project "A" that was created from a copy of another database in project "B". This was achieved with the Cloud Firestore managed export and import service from GCP.

When trying to load documents from a specific collection from the database in project "A", an error is raised:

ValueError                                Traceback (most recent call last)
<ipython-input-16-935f61174138> in <module>()
      1 collection_docs = db.collection("<collection>").stream()
----> 2 for doc in collection_docs:
      3   print(doc.id)

5 frames
/usr/local/lib/python3.7/dist-packages/google/cloud/firestore_v1/_helpers.py in reference_value_to_document(reference_value, client)
    245     if document._document_path != reference_value:
    246         msg = WRONG_APP_REFERENCE.format(reference_value, client._database_string)
--> 247         raise ValueError(msg)
    248 
    249     return document

ValueError: Document 'projects/<project B>/databases/(default)/documents/<another collection>/<document>' does not correspond to the same database ('projects/<project A>/databases/(default)') as the client.

Note that <another collection> is referenced in documents from <collection>

Why the reference keeps pointing to the old database?

UPDATE: It turns out the initial export was correct and the references properly updated, but after that, while updating other fields using a buggy Python script, the references got unnecessarily and incorrectly "updated" to the old project.

The correct way to obtain this field would be:

db.document(<document reference>.path)

Still this question remains:

Is it possible to load the documents without bringing the reference?

Here is the Python snippet that produces the error:

cred = credentials.ApplicationDefault()
firebase_admin.initialize_app(cred, {
  'projectId': <project A>,
})

db = firestore.client()

collection_docs = db.collection("<collection>").stream()
for doc in collection_docs:
  print(doc.id)
google-cloud-firestore python
1 Answers

You can use the Cloud Firestore managed export and import service to export all documents or just specific collections. The data exported from one Cloud Firestore database can be imported into another Cloud Firestore database across projects. The Cloud Firestore managed export and import service is available through the gcloud command-line tool and the Cloud Firestore API (REST, RPC).

Here is the documentation, explaining the same.

Key points to note when importing data:

  • When you import data, the required indexes are updated using your database's current index definitions. An export does not contain index definitions.
  • Imports do not assign new document IDs. Imports use the IDs captured at the time of the export. As a document is being imported, its ID is reserved to prevent ID collisions. If a document with the same ID already exists, the import overwrites the existing document.

Also, I would like you to go through this answer where a community member has explained how to use a value from the Firestore DB of one primary project in order to query the Firestore DB of a secondary project using references. Try and see if you could make it work too without errors, as it is very much possible to point a document from another project in Firestore.

UPDATE

I reproduced your case by creating two projects A and B. In my case, Project A has a Firestore database, with all the data which I am exporting to project B. I followed this documentation for moving the data between projects and it was successfully done. Now after this, I created a service account in Project B and gave the Owner, Service Account Token Creator permissions. Generated a private key file and set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the file path of the JSON file that contains the service account key like this :

export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account-file.json"

After that I wrote the below Python script which worked fine for me and it printed the document id of my Firestore database( which was exported from Project A and imported to Project B) without any errors.

firebase.py

import firebase_admin
from firebase_admin import auth,firestore
from firebase_admin import credentials
cred = credentials.ApplicationDefault()
firebase_admin.initialize_app(cred, {
 'projectId': "projectB’sID",
})
db = firestore.client()
collection_docs = db.collection("collection-name").stream()
for doc in collection_docs:
 print(doc.id)

requirements.txt

firebase_admin=5.0.2
user_158033650
Related