Java Open BufferedWriter only once but rewrite content many times

First of all, an expression like writeDir.concat("/").concat("metadata.txt") is reducing readability and performance. A straight-forward writeDir + "/" + "metadata.txt" will provide better performance. But since you’re constructing a string merely for constructing a Path, it’s even more straight-forward not to do the Path’s job in your code but rather use Paths.get(writeDir, "metadata.txt").

You can not rewind a BufferedWriter but you can rewind a FileChannel. Therefore, to keep the channel open and rewind it when needed, you have to construct a new writer after rewinding:

public static void writeMetaData(FileChannel ch, JSONObject jsonObj) throws IOException {
    ch.position(0);
    if(ch.size() > 0) ch.truncate(0);

    Writer w = Channels.newWriter(ch, StandardCharsets.UTF_8.newEncoder(), 8192);
    w.write(jsonObj.toString());
    w.flush();
}

try(FileChannel ch = FileChannel.open(Paths.get(writeDir, "metadata.txt"),
        StandardOpenOption.WRITE, StandardOpenOption.CREATE)) {

    for(int i = 0; i < 100000; i++) {
        // do Something;
        if(i % 100 == 0) {
            writeMetaData(ch, jsonObject);
        }
    }
}

It’s important that the use of the Writer ends with flush() to force the write of all buffered data, but not close() as that would also close the underlying channel. Note that this code does not wrap the writer into a BufferedWriter; encoding text as UTF-8 is already a buffered operation and by requesting a larger buffer for the encoder, matching BufferedWriter’s default buffer size, we get the same effect of buffering without the copying overhead.

Since writing is not an end in itself, there’s a question left regarding your reading side. If the reader is trying to read the data in some intervals, there’s the risk of overlapping with the write, getting incomplete data.

You could use

public static void writeMetaData(FileChannel ch, JSONObject jsonObj) throws IOException {
    try(FileLock lock = ch.lock()) {
        ch.position(0);
        if(ch.size() > 0) ch.truncate(0);

        Writer w = Channels.newWriter(ch, StandardCharsets.UTF_8.newEncoder(), 8192);
        w.write(jsonObj.toString());
        w.flush();
    }
}

to lock the file during the write. But depending on the system, file locking might not be mandatory but only affect readers also trying to get a read lock.

When you use JDK 11 or newer, you may consider using

for(int i = 0; i < 100000; i++) {
    // do Something;
    if(i % 100 == 0) {
        Files.writeString(Paths.get(writeDir, "metadata.txt"), jsonObject.toString());
    }
}

which clearly wins on simplicity (yes, that’s the complete code, no additional method required). The default options do already include the desired StandardCharsets.UTF_8 and StandardOpenOption.TRUNCATE_EXISTING.

While it does open and close the file internally, it has some other performance tweaks which may compensate. Especially in the likely case that the string consists of ASCII characters only, as the implementation will simply write the string’s internal array directly to the file then.

A Stream does not allow to go back and rewrite content. A way to achieve what you want is using a RandomAccessFile. Its setLength() method will truncate the file if you pass 0.

Here is a simple example:

import java.io.*;

public class Test
{
    public static void updateFile(RandomAccessFile raf, String content) throws IOException
    {
        raf.setLength(0);
        raf.write(content.getBytes("UTF-8"));
    }

    public static void main(String[] args) throws IOException
    {
        try(RandomAccessFile raf = new RandomAccessFile("metadata.txt", "rw"))
        {
            updateFile(raf, "progress: 100");
            updateFile(raf, "progress: 200");
        }
    }
}

File operations are typically buffered by the underlying kernel, and so you're unlikely to see much of a performance benefit by keeping an open file descriptor for this kind of low throughput application.

Keeping your code as a single operation that gets to leave no state after it finishes, rather than designing it as a continuous rewindable stream, makes for an elegant, simple, and unless you've specifically requested synchronous IO, then also sufficiently performant implementation that gets to benefit from the optimizations of all of the layers that sit beneath it.

When you do get measurable impedance to performance by this, which I suspect you never will, you could use the RandomAccessFile API, or go unnecessarily lower level by using FileChannel as others already specified.

I think you shouldn't compromise the simplicity/elegance of your design for this kind of micro-optimization, which in the grand scheme of things, is guaranteed to be insignificant (one tiny write operation per 100 jobs processed).