How to read gz file line by line in Perl6

user_3973175 0
Viewed 354

I'm trying to read a huge gz file line by line in Perl6.

I'm trying to do something like this

my $file = 'huge_file.gz';
for $file.IO.lines -> $line {
    say $line;
}

But this give error that I have a malformed UTF-8. I can't see how to get this to read gzipped material from the help page https://docs.perl6.org/language/unicode#UTF8-C8 or https://docs.perl6.org/language/io

I want to accomplish the same thing as was done in Perl5: http://blog-en.openalfa.com/how-to-read-and-write-compressed-files-in-perl

How can I read a gz file line by line in Perl6?

thanks

raku
3 Answers

I would recommend using the module Compress::Zlib for this purpose. You can find the readme and code on github and install it with zef install Compress::Zlib.

This example is taken from the test file number 3 titled "wrap":

use Test;
use Compress::Zlib;

gzspurt("t/compressed.gz", "this\nis\na\ntest");

my $wrap = zwrap(open("t/compressed.gz"), :gzip);
is $wrap.get, "this\n", 'first line roundtrips';
is $wrap.get, "is\n", 'second line roundtrips';
is $wrap.get, "a\n", 'third line roundtrips';
is $wrap.get, "test", 'fourth line roundtrips';

This is probably the easiest way to get what you want.

user_8040050

use the read-file-content method in the Archive::Libarchive module, but i don't know if the method read all lines into memory at once:

use Archive::Libarchive; 
use Archive::Libarchive::Constants;

my $a = Archive::Libarchive.new: operation => LibarchiveRead, file => 'test.tar.gz';
my Archive::Libarchive::Entry $e .= new;

my $log = '';
while $a.next-header($e) {
    $log = get-log($a,$e) if $e.pathname.ends-with('.txt');
}

sub get-log($a, $e) {
    return $a.read-file-content($e).decode('UTF8-C8');
}
user_58295260

If you are after a quick solution you can read the lines from the stdout pipe of a gzip process:

my $proc = run :out, "gzip", "--to-stdout", "--decompress", "MyFile.gz"

for $proc.out.lines -> $line {
    say $line;
}

$proc.out.close;
user_17722200
Related