I have Flutter web application where I am using Cloud Firestore for backend. My database collection structure is:
event -> eventAttendee -> eventAttendeeHistory
So I will have multiple events and each event can have multiple eventattendees. eventAttendeeHistory is basically I am taking the backup of eventAttendee whenever document changes in eventAttendee collection. Similarly I have eventHistory collection and so on.
I have added "lastUpdatedAt" field in each document which will have FieldValue.ServerTimestamp() assigned.
Everything works smoothly and able to interact the read and write operation.
My application navigation flow is:
loginPage -> landingPage -> eventsListPage -> eventAttendeePage.
So basically when user login, navigate to landing page then from here user navigates to eventListPage where user can select which event user wants to see eventAttendeeDetails. Here event refers to active/get together etc. For example User A arranges get together and then A invites 100 people for this event. All those 100 people will be considered as a eventAttendees. so I will create 100 documents for the particular event(1 document for each attendee).
eventAttendee page allows user to add new attendee, some notes and additional details etc in it.
So inorder to save the firebase read cost, I have enabled Cloud Firestore realtime listener as below where I listen based on lastUpdateAt field. basically, I am trying to fetch the document from server which is greater than the particular eventAttendee local cache latest document lastUpdatedAt field value.
Below code are in onInit stateful class on eventAttendeePage:
StreamSubscription<QuerySnapshot>? eventAttendeeCollectionlistener;
Query<Map<String, dynamic>> queryReference;
//* get the latest lastUpdatedAt timestamp record from local for the particular eventAttendee collection
final latestEventAttendeeEntryFromLocalCache =
await firebaseAccessApiService.firestoreApiServiceInstance
.collection(CollectionName.event)
.doc(event.eventDocId!)
.collection(CollectionName.eventAttendee)
.orderBy('lastUpdatedAt', descending: true)
.limit(1)
.get(const GetOptions(source: Source.cache));
Then here I am checking whether any latest document available in local. This is to handle whether its very first time or not. if no document found then fetch anything >= null. If available then fetch only the document that are greater than latest lastupdatedAt document of local cache.
if (latestEventAttendeeEntryFromLocalCache.docs.isNotEmpty) {
print(
'Inside the eventAttendee - valid last updated document with listener');
final eventAttendeeEntryLocalData = EventAttendeeEntry.fromMap(
latestEventAttendeeEntryFromLocalCache.docs.first.data());
print(
'Event attendee entry data - ${eventAttendeeEntryLocalData.toString()}');
if (eventAttendeeEntryLocalData.lastUpdatedAt != null &&
eventAttendeeEntryLocalData.lastUpdatedAt is Timestamp) {
final localCacheLastUpdateAt =
eventAttendeeEntryLocalData.lastUpdatedAt;
queryReference = firebaseAccessApiService.firestoreApiServiceInstance
.collection(CollectionName.event)
.doc(event.eventDocId!)
.collection(CollectionName.eventAttendee)
.orderBy('lastUpdatedAt', descending: true)
.where(
'lastUpdatedAt',
isGreaterThan: localCacheLastUpdateAt,
);
} else {
throw Exception(
'eventAttendee listener setup is failed because lastUpdatedAt is not valid');
}
} else {
//* control will come here when no record found in local cache for this collection. so we listen all the records from server.
print('Setting up the event attendee listener with >= null');
queryReference = firebaseAccessApiService.firestoreApiServiceInstance
.collection(CollectionName.event)
.doc(event.eventDocId!)
.collection(CollectionName.eventAttendee)
.orderBy('lastUpdatedAt', descending: true)
.where(
'lastUpdatedAt',
isGreaterThanOrEqualTo: null,
);
}
Here listener setup in onInitMethod:
eventAttendeeCollectionlistener =
queryReference.snapshots(includeMetadataChanges: true).listen(
(event) {
print('inside the event attendee listener');
try {
if (event.docChanges.isNotEmpty) {
for (var change in event.docChanges) {
if (change.doc.metadata.hasPendingWrites == false &&
change.doc.metadata.isFromCache == false) {
//* here i am notifiying listener for app functionality.
}
}
}
} catch (e) {
print('Exception occured');
}
},
then in onDispose method, i am cancelling the subscription.
await eventAttendeeCollectionlistener?.cancel();
print('FirestbaseFirestoreListeners is cancelled');
All good so far and logic works fine. It fetches the document from server only the document that are not available in the local for the particular eventAttendee collection. so when user enter into eventAttendeePage, listener picks up the latest document from eventAttendee collection for the particular event and pickup any greater document from server and sync to local cache. when user goes out from this page, it cancel the subscription. This all normal behavior in happy path.
Now problem is, assume there are 2 users (user-1 & user-2) are login in the web application. for ex, user-1 login in windows-1 machine and user-2 login into windows-2 machine. I am using firebase authentication for login. once logged in, assume both user goes to offline (turn off their internet) and then start adding new document into eventAttendee collection from eventAttendee page.
Assume this is the local timestamp of document:
user1: 9.00 (hour.minute) - doc1
user2: 9.02 (hour.minute) - doc2
Now, assume:
user-11 turns on the internet at - 9.05 timestamp.
so doc1 from user-1 machine merge to server with server timestamp of 9.05 timestamp.
But user-2 still in offline, so doc1 from user-1 is not reflected to user-2 machine. similarly doc2 from user-2 is not reflected in server and hence its not reflected in user-1 machine.
Now assume, user-2 refresh the web browser (assume user-2 is in eventattendee page when refresh). according to the my application behavior I push the user-2 to login screen to login back. So it navigates to login screen. Since user-2 is offline, now user-2 has to enable internet to login again. So he does the login. So as soon as the user-2 login, doc2 is automatically merged with server(this is expected) and also reflected in user-1 machine as well. Assume this happens at 9.10 timestamp. so
doc2 from user-2 is stored in server with 9.10 timestamp.
But the problem here is, if user-2 navigates to eventAttendee page for the same event, user-2 does NOT see doc1 from user-1 because user-1 doc1 timestamp(9.05) is lesser than doc2(9.10) (this is my assumption).
My observation here is, as soon as user-2 connects internet and login, Cloud Firestore automatically sync the local to server and hence user-2 local doc2 timestamp also updated to 9.10 in this case. So when user-2 enter into eventAttendee page, the local doc timestamp records shows 9.10. And hence, according to the listener query, its not fetching user-1 doc1 from server and not syncing to user-2 local machine because doc1 timestamp is 9.05.
Question is, how can we solve this problem. I can't put the listener at app level because entering into eventAttendee page is access based and listener query is event & corresponding eventattendee collection level. So I can't put them in app level. Thats why I have put the listener at each eventAttendee collection level. Local doc timestamp will vary for each event because some of the eventAttendee collection are untouched by user. So we can't use some app level timestamp to refresh all the collections. I feel it's also unnecessary. Even if I do, then each time when user enter into event attendee page, it's going to read some unnecessary records from server if i us app login timestamp.
Can someone please share some suggestion to solve this? Appreciate your response. In order to explain the scenario in detail, I have written this long thread.