How to read a specific element from a binary file in Julia?

user_10988530 0
Viewed 150

I have a binary file. If I want to read all the numeric data in an array at once, the code is below:

y = Array{Float32}(undef, 1000000, 1);

read!("myfile.bin", y)

I will get an array y, y is 1000000*1 array{Float32, 2}.

My question is that, I don't want to read all the data in an array at once since it will use a lot of memory. I want to read a specific element in the binary file each time. For example, I only want to read the third element in the binary file which is the third element in array y. How can I do it?

julia
2 Answers

If you just want to read a single element, you don't need to read into an array:

io = open("myfile.bin", "r") # open file for reading
Nbytes = sizeof(Float32) # number of bytes per element
seek(io, (3-1)*Nbytes) # move to the 3rd element
val = read(io, Float32) # read a Float32 element
close(io)

BTW: if you want an array for your data, you should probably use a 1000000 length Array{Float32, 1} instead of a size 1000000x1 Array{Float32, 2}:

y = Array{Float32}(undef, 1000000)
# or
y = Array{Float32, 1}(undef, 1000000)
# or
y = Vector{Float32}(undef, 1000000)
user_27498650

Alternatively, you could mmap the file to access it as an array:

fd = open("myfile.bin")
y = Mmap.mmap(fd, Vector{Float32}, 10000000)
println(y[3])

This will only use virtual memory, but no RAM. You can also make it writeable, too.

user_13462760
Related