Read Specific Lines From Big File Fast with Low Memory Usage

user_19514033 0
Viewed 36

I got inspiration here to read line from specific line of file.

But when I tested it to get range of line from big file: I got 2 different result

Here's the benchmark result reading 100 lines from 10mb file:

Function v1 via file():  in 35ms with memory usage 12.00Mb
Function v2 via SplFileObject: in 956ms with memory usage 2.00Mb

My question, is there other way to do this so its fast like using file() but with low memory like using SplFileObject?

My current functions:

function get_line_content_range_v1($line_number_start, $line_number_end) {

        $content = array();

        $data = file('10mb.txt');
        for($i = $line_number_start; $i <= $line_number_end; $i++) {
            $content[] = $data[$i];
        }

        return $content;

}

function get_line_content_range_v2($line_number_start, $line_number_end) {

        $content = array();

        $file = new SplFileObject("10mb.txt", "r");
        for($i = $line_number_start; $i <= $line_number_end; $i++) {
            $file->seek($i);
            $content[] = $file->current();
        }

        return $content;

}
php
1 Answers

Use a generator to save memory. There is no need to have all contents in RAM.

function get_line_content_range_v3($line_number_start, $line_number_end)
{
    $filehandle  = fopen('10mb.txt', 'r');
    $line_number = 0;
    while (++$line_number <= $line_number_end) {
        $line = fgets($filehandle);
        if ($line_number < $line_number_start) {
            continue;
        }
        yield $line;
    }
    fclose($filehandle);
}

foreach (get_line_content_range_v3(12, 15) as $line) {
    echo $line;
}
user_26457130
Related