Not receiving document updates on network disconnect and reconnect in Firestore Java admin SDK

user_5916957 0
Viewed 754

I am setting up a watch for a query in Firestore in my local PC using Java admin SDK. If the network disconnects and reconnects then the updates are no longer received by my local machine. I have observed that if the network reconnection happens within a short interval (< 2 - 3 mins) then it is fine. But if it is longer then this issue happens. Also, no error/exception is notified in the Listener callback so that I can set up a watch again for the updates. My PC is not behind a proxy so it cannot be a proxy issue.

Please help me in debugging this issue.

Edit:

Looks like this is a bug in the SDK. I enabled the SDK logs and tried the following experiments:

  • Scenario 1:

    1. Watch for query updates
    2. Client sets up a GRPC connection with the server.
    3. Looks like server is sending a keep alive(?) every minute to the client but no keep alive from the client to server
    4. Disconnect the network
    5. Reconnect within a minute, the query connection is still active and document updates arrive as expected
    6. Disconnect the network
    7. Reconnect after 5 minutes
    8. Here my guess is that the server would have reset the GRPC connection from its end since it cannot reach the client but the SDK doesn't know about this. It is still expecting server to send the keep alive.
    9. No document updates are received at the client side (as expected)
    10. Issue a new query and watch for its updates.
    11. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Now the document updates start coming for both the queries.

  • Scenario 2:

    1. Watch for query updates
    2. Disconnect the network for about 5 minutes
    3. Issue a new query before the network is reconnected.
    4. SDK tries to send the query over the same GRPC connection it had established earlier, realizes that the connection is closed, opens a new GRPC connection. Even this fails but the connection attempt is retried every minute.
    5. Reconnect the network.
    6. Connection attempt of SDK succeeds and the updates start coming for both the queries.
google-cloud-firestore firebase java
2 Answers

The GRPC keepalive has to be enabled for the client to send keepalive to the server and detect any connections which are closed from the server side. This can be done by providing TransportChannelProvider in the FirestoreOptions while initializing the FirebaseApp. The code snippet is given below:

InstantiatingGrpcChannelProvider channelProvider =
    InstantiatingGrpcChannelProvider.newBuilder()
        .setKeepAliveTime(Duration.ofSeconds(60L))
        .setKeepAliveTimeout(Duration.ofMinutes(5L))
        .build();

FirestoreOptions firestoreOptions = FirestoreOptions.newBuilder()
    .setChannelProvider(channelProvider).build();

FirebaseOptions options = new FirebaseOptions.Builder()
    .setCredentials(credentials).setFirestoreOptions(firestoreOptions)
    .setConnectTimeout(5000).setReadTimeout(5000).build();
FirebaseApp firebaseApp = FirebaseApp.initializeApp(options);

Firestore firestore = FirestoreClient.getFirestore(firebaseApp);
user_59169570

When you are listening for changes in a Cloud Firestore database and you have some network disconnects, unfortunately there isn't much you can do. You don't have any control on how Firebase Firestore SDK manages its connections.

The reason that the retries aren't happening so quickly as you expect is because the code that performs the retries is unsing a so called exponential backoff algorithm. This means that this code prevents all the retries that can happen on user's device so quickly in favor of performance. Too many retries can also affect the user by consuming too much bandwith of the his data plan.

As a conclusion, this means that it may take some time to regain the connection and you should be patient about this.

user_52468850
Related