Notify when TextIO is done writing a file

I'm working on a scenario with two pipelines using DataFlow in Google Cloud:

Pipeline A runs in streaming mode continuously creating files in Google-Storage based on hourly-windows and some sharding like this:

data.apply(TextIO.write().to(resource.getCurrentDirectory())
            .withFilenamePolicy(new PerWindowFiles(prefix))
            .withWindowedWrites()
            .withNumShards(42));

Pipeline B works in batching mode loading those files regularily for further processing, e.g. every hour.

Here's the Problem: which files can the pipeline B savely load from GS?

• All of them -> probably not a good idea in case A is not done writing some of them and we'll get corrupted files.

• based on time (like load only the files that are at least 2h old) -> will also cause issues in case A is late

• some way of creating "done"-flags in A which tell B which files are done.

• somehow get notified when a window's final pane is done processing -> haven't found a way to do that.

I would like the third approach, but couldn't find a way of determining when TextIO is actually done writing a file without waiting for the pipeline to finish.

The Writer of TextIO does not return another PCollection. One way would be to override the finalize method of the FileBasedSink.WriteOperation which is created somewhere inside TextIO and requires copying the whole class and eventually building a custom Sink. This is overkill in my opinion.

Anyone has ideas for an easier solution or experience how to achieve this?