I regularly deal with different binary file formats in our Java EE application suite, i.e. reading into some class structure and writing them back. I need to have operations like:
- read single byte / short / int / long, sometimes different endianness (little/big)
read single bits in flags, i.e.
| uint16_t | 4 bits | 4 bits | uint16_t |should become something like
a = stream.readUint16(); byte tmp = stream.readUint8(); b = (tmp & 0xf0) >> 4; c = (tmp & 0xf) d = stream.readUint16();read strings in different encodings, sometimes dynamic length strings with something like a
\0ending- seeking in a file (to find data dictated by some offsets read), knowing current position, knowing how much have I left to parse in current data block
- last, but not least, it should be fast; at least not an order of magnitude slower than declaring a
typedef structin C, reading it as a block and typecasting it in memory
So far I've analyzed my options and found that there are:
RandomAccessFile— best one in standard Java, has proper seeking and position methods, string read, etc, but sometimes unbearably slow due to lack of buffering in operations like ; also no bit-level access to the stream and no different endianness supportFileInputStream— can only read individual bytes, one has to reconstruct primitive datatypes by hand; no seeking*Readerinteraces — basically, can only read bytes and array of bytes, can skip, mark and reset, but they tend to leak memory if seeking is done multiple times asreset(); skip(seekAmount);- https://github.com/raydac/java-binary-block-parser — almost exactly what I'm looking for — i.e. a declarative specification of a format and then, voila, I've got classes, but it's essentially an interpreter, thus there are 2 major problems: (a) it's slow in high-demanding environments, (b) there are multiple type safety problems with runtime-generated reflection-like style
- http://preon.codehaus.org/ — has lots of great reviews, but seems to be no longer developed, site is down :(
I've searched Google, I've searched StackOverflow. This question — How to parse/encode binary message formats? — addresses the same issue, but with weird non-aligned bits requirement which I don't have.
So, questions is — am I overlooking something and there are any better solutions for this problem that address all what I've mentioned?