Faster way to write an integer vector to a binary file with C++?

user_11396195 0
Viewed 176

I currently have the following method for writing vector<int> objects to a binary file.

void save_config(std::string fname, std::vector<int> config) {
    std::ofstream out(fname);
    for (auto&& item : config)
        out << item;
}

The data I need to save, however, is on the order of 60 MB and takes about 5 seconds to write with this function. Furthermore, I have to write a binary file for each iteration of an algorithm I am running, and a single iteration for an input size which generates data on this order of magnitude is probably about 500 milliseconds.

I can mask the write time behind the algorithm's execution time but, with this difference in runtime, it won't really matter. Is there any way to improve my save_config function? Also, I'm using a binary file because I've ready that it is the fastest format to write to; but the specific format does not matter so, if someone has an alternate suggestion, I would be happy to hear it.

stdvector binaryfiles c++
1 Answers

Whether or not it will result in a significantly faster operation will have to be tested, but the following use of the write() function avoids iterating through the vector:

#include <fstream>
#include <vector>

void save_config(std::string fname, std::vector<int> config)
{
    std::ofstream out(fname, std::ios_base::binary);
    uint64_t size = config.size();
    out.write(reinterpret_cast<char*>(&size), sizeof(size));
    out.write(reinterpret_cast<char*>(config.data()), size * sizeof(int));
}

Note that I have also included a 'prefix' to record the size of the vector, so that this can be determined when later reading the data from the file; I have used a fixed-size type (64-bits) for this to avoid possible issues with platforms that have a 32-bit size_t type (you should perhaps consider using a fixed-size integer type, such as int32_t, to avoid similar issues).

user_108710730
Related