How to increase timeout for Firestore SDK query stream

user_12284422 0
Viewed 759

I'm trying to build an application based on GCP Firestore and therefore I am using the google-cloud-firestore library. I want to stream my query results via streaming API of Query but getting an StatusRuntimeException after ~60s. Seems like the operation timed out. Where can I increase this timeout?

What I finally want to do is building a stream with Flux, which streams a huge amount of data out of Firestore based on a query.

I just tried to find a possibility to increase the timeout via FirestoreOptions but didn't find a working solution for that. What I can see is, that somewhere during initialization of the stream an rpcTimeout of 60s has been set. But I'm not quite sure that this is the right one. Also I didn't find the location to set it.

Method I'm using:

com.google.cloud.firestore.Query
Query.stream(@Nonnull final ApiStreamObserver<DocumentSnapshot> responseObserver)

Exception after ~60s:

com.google.api.gax.rpc.UnavailableException: io.grpc.StatusRuntimeException: UNAVAILABLE: The datastore operation timed out, or the data was temporarily unavailable.
    at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:69)
    at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72)
    at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60)
...

Edited

Just found out, that it is possible in general to overwrite default settings for timeouts but not for streaming methods. As the respective method in FirestoreSettings states:

/**
* Applies the given settings updater function to all of the unary API methods in this service.
*
* <p>Note: This method does not support applying settings to streaming methods.
*/

That is really annoying, as I didn't have intentions to create a stream based on paging via cursors.

google-cloud-firestore java
1 Answers

The only way I found to control the timeout was like this:

public List<QueryDocumentSnapshot> getAllDocs() {
    try {
        ApiFuture<QuerySnapshot> querySnapshot = db().collection("MyDocs").get();
        return querySnapshot.get(5, TimeUnit.MINUTES).getDocuments();
    } catch (InterruptedException | ExecutionException | TimeoutException e) {
        e.printStackTrace();
    }
    return null;
}
user_56265680
Related