How to write to a file from end to start in c++

Viewed 250

I have a function that computes the result I need on the fly (end result being ~50MB text files containing only chars), but in reverse order.

To illustrate: My function will compute "5, 4, 3, 2, 1, ...", but I require "1, 2, 3, 4, 5" to be written to a file.

I would like to write this to an output file, on the fly, from end to start.

Currently I'm buffering, reversing, and writing to a file with std::ofstream::binary, but I need to cut down the associated time and more importantly space overhead.

What is the most efficient way to do this?

Thank you.

Edit: output size is known.

2 Answers

Actually, writing files from end to start is inefficient solution, because you loose some time on file pre-allocation and seeking. Better just to write array from end to start to file.

I don't know exactly how is written your program, so my answer could be not so accurate as you want, but I assume it could be done something like that:

#include <iostream>
#include <vector>
#include <ostream>
#include <fstream>


// Variant 1
void writeMyData(const std::vector<int>& data, std::ofstream& stream)
{
    for (auto it = data.rbegin(); it != data.rend(); it++)
        stream.write((const char*)&*it, sizeof(int));
}


// Variant 2
void writeMyData2(const int* data, int length, std::ofstream& stream)
{
    for (int i = length - 1; i >= 0; i--)
        stream.write((const char*)(data + i), sizeof(int));
}


int main()
{
    std::vector<int> yourBuffer = {5, 4, 3, 2, 1};
    std::ofstream file;
    file.open ("data.bin", std::ios::binary);
    
    // Write: 1, 2, 3, 4, 5
    // Variant 1
    writeMyData(yourBuffer, file);
    
    // Variant 2
    //writeMyData2(yourBuffer.data(), yourBuffer.size(), file); 
    
    file.close();
    return 0;
}

So, you can use reverse iterators (rbegin, rend instead of begin, end) or just iterate loop in reverse order.

Update: Or you can reverse your data in RAM inplace, and then just write all data at once. Probably, it will be the most efficient way.

std::reverse(std::begin(yourBuffer), std::end(yourBuffer));
stream.write((const char*)yourBuffer.data(), sizeof(int) * yourBuffer.size());

Be attentive with sizeof(int) - probably you have char or some another type.

I understand you requirements like this:

  • Write data in binary form (although you are talking about text files somehwere)
  • You want to store plain chars
  • You know the number of characters to write in advance
  • And then write the the first element at the end, then one before and so on.

If you realy do not want to use buffers, then you must use seekp. But already now some hint: This will be a very slow solution.

Anyway, how to do? So:

  • First open a file for output and check, if it could be opened
  • Calculate the offset position of the last charachter in the output file
  • Seek to this position and write
  • Decrement offset

Seek and write will look like this

fileStream.seekp(offset--).put(testData[index]);

Since I do not have your calculation function, I created some dummy data. This needs to be adapted by you.

See the following example snippet:

#include <iostream>
#include <fstream>
#include <array>
#include <numeric>

// The known size of test data
constexpr size_t OutputSize = 50'000U;
// Some simple test data
static std::array<char, OutputSize> testData{};

int main() {

    // You will calculate the data in a different part of your code
    // I jsut fill the array with some data
    std::iota(testData.begin(), testData.end(), 0);

    // Open the file and check, if it could be opened.
    if (std::ofstream fileStream{ "r:\\test.bin", std::ofstream::binary }; fileStream) {

        // Caculate position offset of last element
        size_t offset{ (OutputSize - 1)};

        // Calculate and write data
        for (size_t index{}; index < OutputSize; ++index) {

            // Your claculation here
            // . . .

            // Seek and write data
            fileStream.seekp(offset--).put(testData[index]);
        }
    }
    return 0;
}

Again. Seeking is very slow.

Related