What's the quickest and most efficient way of reading the last line of text from a [very, very large] file in Java?
What's the quickest and most efficient way of reading the last line of text from a [very, very large] file in Java?
Have a look at my answer to a similar question for C#. The code would be quite similar, although the encoding support is somewhat different in Java.
Basically it's not a terribly easy thing to do in general. As MSalter points out, UTF-8 does make it easy to spot \r or \n as the UTF-8 representation of those characters is just the same as ASCII, and those bytes won't occur in multi-byte character.
So basically, take a buffer of (say) 2K, and progressively read backwards (skip to 2K before you were before, read the next 2K) checking for a line termination. Then skip to exactly the right place in the stream, create an InputStreamReader on the top, and a BufferedReader on top of that. Then just call BufferedReader.readLine().
Using FileReader or FileInputStream won't work - you'll have to use either FileChannel or RandomAccessFile to loop through the file backwards from the end. Encodings will be a problem though, as Jon said.
as far as I know The fastest way to read the last line of a text file is using FileUtils Apache class which is in "org.apache.commons.io". I have a two-million-line file and by using this class, it took me less than one second to find the last line. Here is the my code:
LineIterator lineIterator = FileUtils.lineIterator(newFile(filePath),"UTF-8");
String lastLine="";
while (lineIterator.hasNext()){
lastLine= lineIterator.nextLine();
}
In C#, you should be able to set the stream's position:
From: http://bytes.com/groups/net-c/269090-streamreader-read-last-line-text-file
using(FileStream fs = File.OpenRead("c:\\file.dat"))
{
using(StreamReader sr = new StreamReader(fs))
{
sr.BaseStream.Position = fs.Length - 4;
if(sr.ReadToEnd() == "DONE")
// match
}
}
To avoid the Unicode problems related to reverting the string (or the StringBuilder), as discussed in Eric Leschinski excellent answer, one can read to a byte list, from the end of the file, revert it to a byte array and then create the String from the byte array.
Below are the changes to Eric Leschinski answer's code, to do it with a byte array. The code changes are below the commented lines of code:
static public String tail2(File file, int lines) {
java.io.RandomAccessFile fileHandler = null;
try {
fileHandler = new java.io.RandomAccessFile( file, "r" );
long fileLength = fileHandler.length() - 1;
//StringBuilder sb = new StringBuilder();
List<Byte> sb = new ArrayList<>();
int line = 0;
for(long filePointer = fileLength; filePointer != -1; filePointer--){
fileHandler.seek( filePointer );
int readByte = fileHandler.readByte();
if( readByte == 0xA ) {
if (filePointer < fileLength) {
line = line + 1;
}
} else if( readByte == 0xD ) {
if (filePointer < fileLength-1) {
line = line + 1;
}
}
if (line >= lines) {
break;
}
//sb.add( (char) readByte );
sb.add( (byte) readByte );
}
//String lastLine = sb.reverse().toString();
//Revert byte array and create String
byte[] bytes = new byte[sb.size()];
for (int i=0; i<sb.size(); i++) bytes[sb.size()-1-i] = sb.get(i);
String lastLine = new String(bytes);
return lastLine;
} catch( java.io.FileNotFoundException e ) {
e.printStackTrace();
return null;
} catch( java.io.IOException e ) {
e.printStackTrace();
return null;
}
finally {
if (fileHandler != null )
try {
fileHandler.close();
} catch (IOException e) {
}
}
}