Why do we need numpy.fromfile if we already have numpy.load?

user_9793316 0
Viewed 1989

numpy.fromfile

According to SciPy documentation

Construct an array from data in a text or binary file. [...] Data written using the tofile method can be read using this function.

So I just follow the instruction and create a file by using tofile

import numpy as np
A = np.random.rand(1000)
A.tofile('binary_file') # saving a dummy binary_file

x1 = np.fromfile('binary_file') # loading the dummy file

print(np.array_equal(A, x1)) # checking if the arrays are the same
>>> True

numpy.load

At the same time, we also have np.save and np.load, which do exactly the save job as tofile and fromfile

np.save('file.npy',A)
x2 = np.load('file.npy')
print(np.array_equal(A, x1))
>>> True

fromfile == np.load?

The two loading methods yield to same result as demonstrated below

print(np.array_equal(x1, x2))
>>> True

Then why do need both functions? Anything that can be done on np.load but not fromfile? Or vice versa?

python-3.x numpy
2 Answers

fromfile == np.load?

No. numpy.load reads files in the NPY format. This is a specific format containing metadata that defines the shape and data type of the numpy array.

numpy.fromfile reads the raw bytes from the file. No metadata is stored in the file. Your example with A happened to work because the default data type assumed by fromfile is float64. Here's an example where it does not work:

In [25]: A = np.array([10, 20, 30, 40])                                                                                                              

In [26]: A.tofile('binary_file')                                                                                                                     

In [27]: B = np.fromfile('binary_file')                                                                                                              

In [28]: B                                                                                                                                           
Out[28]: array([4.9e-323, 9.9e-323, 1.5e-322, 2.0e-322])
user_12173580

fromfile is the low-level function that is numpy calls within load:

  • np.load (aka np.lib.npio.load) calls format.read_array on line 452. Looking at the source for load, you can see it tries to guess what the right way to load a bunch of different binary file types is.
  • np.lib.format.read_array calls fromfile on line 738. This loads from the npy file format, which is a small header plus the raw binary data
  • fromfile is a python wrapper around PyArray_FromFile, which is a weird function that either loads raw binary data (the useful version), or loads text data in yet another different way.
user_1024410
Related