How to write/serialize lucene's ByteBuffersDirectory to disk?

Viewed 151

How one would write a Lucene 8.11 ByteBuffersDirectory to disk?
something similar to Lucene 2.9.4 Directory.copy(directory, FSDirectory.open(indexPath), true)

2 Answers

You can use the copyFrom method to do this.

For example:

You are using a ByteBuffersDirectory:

final Directory dir = new ByteBuffersDirectory();

Assuming you are not concurrently writing any new data to that dir, you can declare a target where you want to write the data - for example, a FSDirectory (a file system directory):

Directory to = FSDirectory.open(Paths.get(OUT_DIR_PATH));

Use whatever string you want for the OUT_DIR_PATH location.

Then you can iterate over all the files in the original dir object, writing them to this new to location:

IOContext ctx = new IOContext();
for (String file : dir.listAll()) {
    System.out.println(file); // just for testing
    to.copyFrom(dir, file, file, ctx);
}

This will create the new OUT_DIR_PATH dir and populate it with files, such as:

_0.cfe
_0.cfs
_0.si
segments_1

... or whatever files you happen to have in your dir.

Caveat:

I have only used this with a default IOContext object. There are other constructors for the context - not sure what they do. I assume they give you more control over how the write is performed.

Meanwhile I figured it out by myself and created a straight forward method for it:

    @SneakyThrows
    public static void copyIndex(ByteBuffersDirectory ramDirectory, Path destination) {
        FSDirectory fsDirectory = FSDirectory.open(destination);
        Arrays.stream(ramDirectory.listAll())
                .forEach(fileName -> {
                    try {
                        // IOContext is null because in fact is not used (at least for the moment)
                        fsDirectory.copyFrom(ramDirectory, fileName, fileName, null);
                    } catch (IOException e) {
                        log.error(e.getMessage(), e);
                    }
                });
    }
Related