How do I read one line at a time in from two files in Julia?

user_11902012 0
Viewed 75

I am working in Julia 1.4. I would like to open two large gzipped files (file1.gz and file2.gz) and then read the first line from file 1, the first line from file 2, do something with these, and then move on to the second line of each file etc. If I nest two for loops, this obviously does not work because it cycles through file2 before moving on to the next line of file1. The files are two big to open all at once.

   handle1 = GZip.open(file1.gz)
   handle2 = GZip.open(file2.gz)
for line1 in eachline(handle1)
for line2 in eachline(handle2)
println(line1,line2)
end
end

Is there a simple solution ?

large-data julia gzip
1 Answers

Yes, you can use zip. You can also manage the eachline iterators yourself, but using zip is easier:

handle1 = GZip.open(file1.gz)
handle2 = GZip.open(file2.gz)

for (line1, line2) in zip(eachline(handle1), eachline(handle2))
    println(line1,line2)
end

close(handle1)
close(handle2)

Don't forget to close your files!

Also, do note that if the two files have a different number of lines, the zip iterator will stop when the first of the two files runs out.

user_109926670
Related