c++ stack overflow due to recursive function how can I improve the data handling

Viewed 135

I'm tackling a exercise which is designed to cause exactly this problem, of overloading the memory. Pretty much I'm loading various file sizes from 1,000 to 5 million lines of entries like this in a txt file (1 line = 1 entry):

SHFIv,aiSdG
PlgNB,bPHoP
ZHWJU,gfwgC
UAygL,Vqvhi
BlyzX,LLbCo
jbvrT,Utblj
...

pretty much every entry has 2 values separated by comma, in my code, I separate these values and try to find another matching value, there are always only 2 exactly matching values and each time 1 value is found the other one with which it is paired points to another pair, and so on until the final one gets found.

For example SHFIv,aiSdG would point to aiSdG,YDUVo.

I know my code is not very efficient, partly due to using recursion, but I could'nt figure out a better way to do the job, so any suggestions on how to possibly improve it to handle larger inputs would be greatly appriciated

#include <iostream>
#include <fstream>
#include <string>
#include <sstream>
#include <map>
#include <unordered_map>
#include <stdio.h>
#include <vector>
#include <iterator>
#include <utility>
#include <functional>
#include <algorithm>

using namespace std;


template<typename T>
void search_bricks_backwards(string resume, vector<T>& vec, vector<string>& vec2) {
    int index = 0;
    for (const auto& pair : vec) {
        //cout << "iteration " << index << endl;
        if (pair.second == resume) {
            vec2.insert(vec2.begin(), resume);
            cout << "found " << resume << " and " << pair.second << endl;
            search_bricks_backwards(pair.first, vec, vec2);
        }
        if (index + 1 == vec.size()) {
            cout << "end of backward search, exitting..." << endl;


        }
        index++;
    }

}



template<typename T>

void search_bricks(string start, vector<T>& vec, vector<string>& vec2) {
    int index = 0;
    for (const auto& pair : vec) {
        //cout << "iteration " << index << endl;
        if (pair.first == start) {
            vec2.push_back(start);
            cout << "found " << start << " and " << pair.first << endl;
            search_bricks(pair.second, vec, vec2);
        }
        if (index + 1 == vec.size()) {
            //search_bricks_backwards(start, vec, vec2);
           // this also gets called on every recursion rather than just once
           // as I originally intended when the forward iteration gets finished
            

        }
        index++;
    }
    
}

template<typename T> // printing function
void printVectorElements(vector<T>& vec)
{
    for (auto i = 0; i < vec.size(); ++i) {
        cout << "(" << vec.at(i).first << ","
            << vec.at(i).second << ")" << endl ;
    }
    cout << endl;
}

vector<string> split(string s, string delimiter) { // filtering function
    size_t pos_start = 0, pos_end, delim_len = delimiter.length();
    string token;
    vector<string> res;

    while ((pos_end = s.find(delimiter, pos_start)) != string::npos) {
        token = s.substr(pos_start, pos_end - pos_start);
        pos_start = pos_end + delim_len;
        res.push_back(token);
    }

    res.push_back(s.substr(pos_start));
    return res;
}


int main()
{
    vector<pair<string, string>> bricks;
    vector<string> sorted_bricks;

    ifstream inFile;
    inFile.open("input-pairs-5K.txt"); // transferring data from .txt to a string

    stringstream strStream;
    strStream << inFile.rdbuf(); 
    string str = strStream.str(); 

    
    

    istringstream iss(str);

    for (string line; getline(iss, line); ) 
     // filtering data from string and dividing on ","
    {

        string delimiter = ",";
        string s = line;
        vector<string> v = split(s, delimiter);
        string s1 = v.at(0);
        string s2 = v.at(1);

        bricks.push_back(make_pair(s1, s2));
    }


    search_bricks(bricks[0].second, bricks, sorted_bricks);
    
    
    //printVectorElements(bricks); 
    
    //for (auto i = sorted_bricks.begin(); i != sorted_bricks.end(); ++i)
        //cout << *i << " "; // this is just to check if vectors have data


    
 
}

Here is link to the 1k test data that works for me (only for the search bricks without backwards searching since it triggers on every recursion) again thanks for any suggestions on how to improve or get rid of the recursion. I don't code in c++ often and don't really know how else to tackle this.

1 Answers

Although implementing non-recursive version of your algorithm is canonical solution, if you really need to solve the problem without code modification, you can increase the stack size by modifying compiler option. ~100Mb will be usually sufficient.

In MSVC : /STACK:commit 104857600

In gcc : --stack, 104857600

Related