Numpy data size twice of the Eigen Matrix size is that the case?

user_5994231 0
Viewed 71

I am generating numpy array with :

p_desct = np.random.uniform(-1, 0.4, [5000000, 512])

Memory size almost ~20G

Same data in Eigen Matrix (C++):

    Eigen::MatrixXf x_im = Eigen::MatrixXf::Random(5000000,512);

MemorySize ~9,6G

Is that the case numpy array doubles the memory usage of same matrix ?

or am I missing something here ?

eigen numpy python c++
2 Answers

The default numpy dtype is float_, but confusingly, this is a double [https://numpy.org/doc/stable/reference/arrays.scalars.html#numpy.float_].

The f suffix in the Eigen data type indicates a 32 bit traditional float, hence being half the size of the 64 bit doubles numpy is using.

Try

np.random.uniform(-1, 0.4, [5000000, 512], dtype=np.float32) and compare.

user_28951820

Numpy arrays use 64-bit floats (often called 'double') by default, while your C++ array uses 32-bit floats. This means your numpy array takes twice as much memory as your C++ one. Specify dtype = np.float32 if you wish to use 32-bit floats.

Also see https://numpy.org/doc/stable/user/basics.types.html for all numpy array data types

user_123371950
Related