How to move a FileHandle's internal position without actually reading data?

user_1885936 0
Viewed 265

I am trying to read a binary file byte by byte in Node using fs.open and FileHandle. Using FileHandle.read(buffer, offset, length, position), I can either:

  1. Leave the position argument blank, in which case every chunk of data read moves the FileHandle's internal position
  2. Supply a position argument, in which case the internal position will remain unchanged

I would like to read a value from a specific position in the middle of the file (using FileHandle.read with a position), and based on that value, sequentially read some other data from another point in the file (using FileHandle.read with no position).

The first part is easy. For the second part, I know that I can:

  1. Maintain a custom position variable and supply it to every read call
  2. Read a certain amount of data from the file just to change the internal position

The first solution is workable but a bit of a hassle, as the structure of that data is quite complicated. The second solution is not really viable as the files get rather large, and the whole point of reading the file byte by byte was to not read large chunks of its content into memory.

I can also build my own abstraction over FileHandle, one which would maintain its own internal position and allow me to alter it, but I'd rather avoid that if I can.

Is there a method that would allow me to skip an arbitrary amount of bytes in a FileHandle, or to manually set its internal position?

fs node.js javascript
1 Answers

The fs library takes a hands-off strategy with the file position, leaving it to libuv and the OS to handle that entirely. So, there is no saved position in the FileHandle object.

You can see the relevant code here where it just does this:

position = bufferOrOptions.position || null;

and just passes that position variable directly through to the native code bindings:

const bytesRead = (await binding.read(handle.fd, buffer, offset, length,
                       position, kUsePromises)) || 0;

So, the fs module itself does not keep track of the file position and thus you can't "set" it in the fs module. Other than hacking on nodejs internals, I don't know of any other way to "set" the file position other than your own suggestion of overriding various methods to keep track of the "current" file position and then being able to set that file position to the "current" value before any API call proceeds that didn't explicitly pass in the file position.

Coming from C++ where this was such a built-in part of the std file library, this has always seemed strange to me in nodejs that there's no provision for it. It appears the perhaps libuv does not even have a sense of the file position if you look at the file operations available in it here so perhaps this is why nodejs follows suit.

Anyway, sorry I couldn't offer any magic, simple answer, but hopefully this explains the state of things in this regard and you can decide how to best proceed.

user_8166200
Related