I want to emulate the functionality of gzcat | tail -n.
This would be helpful for times when there are huge files (of a few GB's or so). Can I tail the last few lines of such a file w/o reading it from the beginning? I doubt that this won't be possible since I'd guess for gzip, the encoding would depend on all the previous text.
But still I'd like to hear if anyone has tried doing something similar - maybe investigating over a compression algorithm that could provide such a feature.
No, you can't. The zipping algorithm works on streams and adapts its internal codings to what the stream contains to achieve its high compression ratio.
Without knowing what the contents of the stream are before a certain point, it's impossible to know how to go about de-compressing from that point on.
Any algorithm which allows you to de-compress arbitrary parts of it will require multiple passes over the data to compress it.
If you have control over what goes into the file in the first place, if it's anything like a ZIP file you could store chunks of predetermined size with filenames in increasing numerical order and then just decompress the last chunk/file.
An example of a fully gzip-compatible pseudo-random access format is dictzip:
For compression, the file is divided up into "chunks" of data, each chunk is less than 64kB. [...]
To perform random access on the data, the offset and length of the data are provided to library routines. These routines determine the chunk in which the desired data begins, and decompresses that chunk. Consecutive chunks are decompressed as necessary."
Well, you can do that if you previously creates an index for each file ...
I've developed a command line tool which creates indexes for gzip files which allow for very quick random access inside them, and it does this interleaved with actions (extract, tail, continuous tail, etc): https://github.com/circulosmeos/gztool
But you can do a tail (-t), and the index will be automatically created: if you're gonna do the same in the future it'll be much quicker, and anyway the first time it will take the same time as a gunzip | tail:
$ gztool -t my_file.gz