Unexpected behavior constructing a java.util.Scanner

user_451600 0
Viewed 136

I have the following file lines.txt

Line1
Line2
Line3

I'm using a Scanner to parse the contents of this file line by line. I have the following setup in LinesReader.java

import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;

class Line {
    Line(String content) {
        this.content = content;
    }
    public String content;

    public String toString() {
        return content;
    }
}

public class LinesReader {
    public static Line buildLine(InputStream is) {
        Scanner scanner = new Scanner(is);
        if (scanner.hasNextLine()) {
            return new Line(scanner.nextLine());
        }
        return null;
    }
    public static Line buildLine(Scanner scanner) {
        if (scanner.hasNextLine()) {
            return new Line(scanner.nextLine());
        }
        return null;
    }

    public static void main(String[] args) throws FileNotFoundException {
        List<Line> lines = new ArrayList<>();
        Line line = null;
        FileInputStream is = new FileInputStream("lines.txt");
        // buildLine(scanner) works as expected
        while ((line = buildLine(is)) != null) {
            lines.add(line);
        }

        System.err.println(lines);
    }
}

The output is

[Line1]

The expected output would be

[Line1, Line2, Line3]

I understand the Scanner implements AutoCloseable, but according to the documentation that would only apply for a try-with-resources construct and not here. Also, when i debug it is says the underlying stream is open. The second call to scanner.hasNextLine() unexpectedly fails.

If I construct the scanner once in main() it works as expected.

My java version is 1.8.0_275

In response to a comment by @Sweeper the scanner seems to buffer up more than what is consumed, the documentation sort of contradicts that.

for hasNextLine()

The scanner does not advance past any input.

for nextLine()

Since this method continues to search through the input looking for a line separator, it may buffer all of the input searching for the line to skip if no line separators are present.

Emphasis mine.

java.util.scanner java
3 Answers

The documentation for hasNextLine()

The scanner does not advance past any input.

is somewhat misleading. It doesn't advance the internal buffer of the scanner, which is obvious, but several kilobytes of the stream is read.

In this case the entire stream is consumed by hasNextLine().

My personal opinion is that this is a defect in the implementation of Scanner. Scanner is designed for convenience and simplicity, not for performance. Wrapping the InputStream in a BufferedInputStream would be sensible and make the usage a a lot simpler.

user_4516000

A Scanner is buffered, and one cannot expect that the underlying (File)InputStream is not read further than what is returned by nextLine. In fact the underlying FileInputStream could be advanced to the end-of-file. So the first Scanner instance could let the FileInputStream at the end-of-file.

Since java 8, it is easier to use Path, Files, Stream.

Path path = Paths.get("lines.txt");
try (Stream<String> in = Files.lines(path, Charset.defaultCharset())) {
    List<Line> lines = in.map(Line::new)
            .collect(Collectors.toList());
    ...
}

The above also automatically closes the file, try-with-resources syntax.

The code is smaller with the new classes.

user_9848230

Try this.

    ...
    FileInputStream is = new FileInputStream("lines.txt");
    while (true) {
        System.err.println(is.available());
        Scanner scanner = new Scanner(is);
        if (scanner.hasNextLine()) {
            lines.add(new Line(scanner.nextLine()));
        } else {
            break;
        }
    }

I got the following.

18
0    <---- FileInputStream is not avaliable for the 2nd Scanner
[Line1]

But if I move the line

    while (true) {
        FileInputStream is = new FileInputStream("lines.txt");
        System.err.println(is.available());
        ...
    }

the loop keeps printing out 18.

user_42271750
Related