How to get rid of incorrect symbols during Java NIO decoding?

user_14190761 0
Viewed 140

I need to read a text from file and, for instance, print it in console. The file is in UTF-8. It seems that I'm doing something wrong because some russian symbols are printed incorrectly. What's wrong with my code?

StringBuilder content = new StringBuilder();

        try (FileChannel fChan = (FileChannel) Files.newByteChannel(Paths.get("D:/test.txt")) ) {

            ByteBuffer byteBuf = ByteBuffer.allocate(16);
            Charset charset = Charset.forName("UTF-8");

            while(fChan.read(byteBuf) != -1) {
                byteBuf.flip();
                content.append(new String(byteBuf.array(), charset));
                byteBuf.clear();
            }

            System.out.println(content);
        }

The result:

Здравствуйте, как поживае��е?
Это п��имер текста на русском яз��ке.ом яз�

The actual text:

Здравствуйте, как поживаете?
Это пример текста на русском языке.
decode nio java
1 Answers

UTF-8 uses a variable number of bytes per character. This gives you a boundary error: You have mixed buffer-based code with byte-array based code and you can't do that here; it is possible for you to read enough bytes to be stuck halfway into a character, you then turn your input into a byte array, and convert it, which will fail, because you can't convert half a character.

What you really want is either to first read ALL the data and then convert the entire input, or, to keep any half-characters in the bytebuffer when you flip back, or better yet, ditch all this stuff and use code that is written to read actual characters. In general, using the channel API complicates matters a ton; it's flexible, but complicated - that's how it goes.

Unless you can explain why you need it, don't use it. Do this instead:

Path target = Paths.get("D:/test.txt");
try (var reader = Files.newBufferedReader(target)) {
    // read a line at a time here. Yes, it will be UTF-8 decoded.
}

or better yet, as you apparently want to read the whole thing in one go:

Path target = Paths.get("D:/test.txt");
var content = Files.readString(target);

NB: Unlike most java methods that convert bytes to chars or vice versa, the Files API defaults to UTF-8 (instead of the useless and dangerous, untestable-bug-causing 'platform default encoding' that most java API does). That's why this last incredibly simple code is nevertheless correct.

user_7686440
Related