As a passionate high-school teacher, I am using Firebase Cloud Firestore & Flutter for my read-only app. My data is about 2000 image.png files with some tags as categories and subcategories. Categories, subcategories and images have an order among themselves. Reason of me using Cloud Firestore is, I can change (= add, delete, update) my categories subcategories and images anytime and my user will be able update with my latest content.
So far everything works fine, my app is fully functional offline, I can synchronize with my content, but if device is online, all the documents in Firestore has been read every time even if there is no change in db. That makes me think that I am doing a huge mistake.
This is the code for what happens when app is opened:
return FutureBuilder<QuerySnapshot>(
future: _getCategories(),
builder: (context, catSnapshot) {
if((!catSnapshot.hasData) && !catSnapshot.hasError) return customProgressIndicator("Categories Synchronizing");
if(_categories.isNotEmpty) _categories.clear();
_categories = catSnapshot.data.documents.map<Category>((document) => Category.fromJson(document.data)).toList();
return FutureBuilder(
future: _getSubCategories(),
builder: (context, subSnapshot) {
if(!subSnapshot.hasData && !subSnapshot.hasError) return customProgressIndicator("Subcategories Synchronizing");
if(_subcategories.isNotEmpty) _subcategories.clear();
for(int i=0; i<_categories.length; i++) {
String catName = _categories[i].name;
List<Subcategory> subcategoriesI = subSnapshot.data.documents.where((document) => document.data["catName"]==catName).map<Subcategory>((document) => Subcategory.fromJson(document.data)).toList();
_subcategories[catName] = subcategoriesI;
}
return RefreshIndicator(
onRefresh: _onRefresh,
child: _categories.length==0 ? whenNoData() : ListView.builder(
itemBuilder: (context, i) => _createCategoryItem(i),
itemCount: _categories.length,
padding: EdgeInsets.all(8.0),
), ); }, ); }, )
This is my main page:
class _CategoryListPageState extends State<CategoryListPage> {
List<Category> _categories = [];
Map<String, List<Subcategory>> _subcategories = {};
Future<QuerySnapshot> _getCategories() async{
var conn=await UserHasConnection();
print("connection: " + conn.toString());
Future<QuerySnapshot> snapshots;
if (widget.isUserPro) {
snapshots = Firestore.instance.collection("categories").where("isPro", isEqualTo: true).orderBy("showOrder").getDocuments();
snapshots.timeout(Duration(seconds: globalTimeOut), onTimeout: () {print("Timeout Category");});
return snapshots;
} else {
snapshots = Firestore.instance.collection("categories").orderBy("showOrder").getDocuments();
snapshots.timeout(Duration(seconds: globalTimeOut), onTimeout: () {print("Timeout Category");});
return snapshots;
}
}
_getSubCategories() async{
Future<QuerySnapshot> snapshots;
if(widget.isUserPro){
snapshots=Firestore.instance.collection("subcategories").where("isPro", isEqualTo: true).orderBy("showOrder").getDocuments();
snapshots.timeout(Duration(seconds: globalTimeOut), onTimeout: () {
print("Timeout Subcategory");
});
return snapshots;
} else { snapshots=Firestore.instance.collection("subcategories").orderBy("showOrder").getDocuments();
snapshots.timeout(Duration(seconds: globalTimeOut), onTimeout: () {
print("Timeout Subcategory");
});
return snapshots;
}
}
Those are my Firestore document collections:
categories
isPro: boolean (there aid free and paid-proUser's)
name: string, name of the category
scCount: int, number of subcategories it has
showOrder: its order among other categories
subcategories
catName: name of the category it belongs to
fcCount: number of image.png it has
imageBytes: subcategories have an image
isPro: boolean, subcategories can also be free or paid
name: subcategories have a name
showOrder: each subcategory has an order among other subcategories in that category
title: title of the subcat, similar to name
flashcards
backBytes: back image base64encoded
backStamp
catName: category of this image belongs to
frontBytes: front image base64encoded
frontStamp
isPro: cards are also free or paid (for search function below)
name
showOrder: order of this card in the subcategory
subName: name of the subcategory that this card belongs to
*** I have separate flascards, flashcards2x, flashcards3x collections for different device screen sizes
Also there is a search function which is done from flashcardContents collection frontContent property:
flashcardContents
frontContent
globalOrder
isPro
name
showOrder
updateTime
updateType: (1-adden, 2-updated, 3-deleted)
Lastly, my FirebaseService functions:
class FlashcardFirebaseService extends IServiceBase {
Future<List<Flashcard>> QueryRatedFlashCards(List<Flashcard> _flashcards,
List<String> flashcardNames, String ratioPostfix, bool isUserPro) async {
var snapshot;
flashcardNames.forEach((flashcardName) {
print("flashcardName: " + flashcardName);
if (isUserPro) {
//For pro user
snapshot = Firestore.instance
.collection("flashcards${ratioPostfix}")
.where("name", isEqualTo: flashcardName)
.snapshots();
} else {
snapshot = Firestore.instance
.collection("flashcards${ratioPostfix}")
.where("name", isEqualTo: flashcardName)
.where("isPro", isEqualTo: false)
.snapshots();
}
snapshot.listen((onData) {
_flashcards.addAll(onData.documents
.map<Flashcard>((document) => Flashcard.fromJson(document.data))
.toList());
});
});
return _flashcards;
}
static Future<List<Category>> SyncCategories(bool isPro, double lastStamp){
Firestore.instance
.collection("categories")
.getDocuments().then((data){
return data;
});
}
static Future<List<Category>> SyncSubcategories(bool isPro){
Firestore.instance
.collection("subcategories")
.getDocuments().then((data){
return data;
}); } }
class FlashcardSearchService extends IServiceBase {
static Future SyncLocalSearchDb() async {
Query baseQuery;
AppConfig appConfig;
appConfig = await AppConfigService.GetAppConfiguration().then((appConfig){
if (appConfig != null) {
var lastSyncDate;
try {
lastSyncDate = appConfig.fcLastSync;
} catch (ex) {}
print("lastSyncDate ---------------------------:"+ (lastSyncDate/1000).toString());
baseQuery = Firestore.instance
.collection("flashcardContents")
.where("updateTime", isGreaterThan: lastSyncDate/1000);
} else {
appConfig = null;
print("appConfig null");
//Create all card contents
baseQuery = Firestore.instance.collection("flashcardContents");
appConfig = new AppConfig();
}
baseQuery.getDocuments().then((query) {
query.documents.forEach((doc) => _syncFlashcardContent(doc));
if(query.documents.length>0)
{
appConfig.fcLastSync = new DateTime.now().millisecondsSinceEpoch.toDouble();
print("new lastSyncDate ---------------------------:"+ (appConfig.fcLastSync /1000).toString());
print("SYNC!!! " + appConfig.fcLastSync.toString());
AppConfigService.UpdateApplicationConfigurationSyncTime(appConfig);
} else {
print("CANT SYNC!!! " + appConfig.fcLastSync.toString());
}
return appConfig;
}).timeout(Duration(seconds: globalTimeOut),onTimeout: (){
print("TimeOutSync");
return appConfig;
});
});
To summarize, I have a flashcards image.png collection for my students to use for studying. My content changes to some extent on a weekly basis and I want my users to catch up with me. Problem is, with my code if device is online all document is read constantly even if there is no change and even if I close the app seconds later I open it. This becomes unaffordable for me.
Sorry for long post, I hope at least you like the code, any idea is valuable and appreciated.
I found out that main cause of my app having that much read count is: "If the listener is disconnected for more than 30 minutes (for example, if the user goes offline), you will be charged for reads as if you had issued a brand-new query." So new question is: Is it possible to change the behavior of this listener? Or is there a way to bypass it? I am feel like my app is not so suitable for Firestore but so far this seems to be only problem!
To summerize: I need a listener that works, for example, once in a day, because my collection will change probably once in a day. Or I need a way to by-pass this. For example when my app goes offline, listener does not work. any options to disable my apps network connection without disabling Firestore offline persistence? Thanks