Insert into unordered_map hanging on runtime

Question

I've run into an issue that I can only describe as completely unfamiliar to me, and I could use some help. For context, I don't know a whole lot about c++ programming and am trying to learn more, so the code I'm currently working on is more educational than anything else. I want to create an unordered_map that contains a string as a key and a vector as the key's associated value. Traditionally, this would be trivial. However, my program is trying to parse json data into this unordered_map data structure and there are no guarantees on the type of each vector being known in advance. The only thing that I can guarantee is that the type of each vector will be a one of the types in the following set of types: string, int, double. In order to try and achieve an unordered_map that functions this way, I've tried to use variant, but currently the variant implementation is leading to a timeout error. I'm hoping to get some advice on the cause of the timeout, and, ideally, how to resolve it.

The code is as follows (minimum example that replicates this issue):


#include 
#include 
#include 
#include 
#include  

using json = nlohmann::json;

int main() {
// map stores col name as string and vector with col type
    std::unordered_map>> mp;

   // input is type nlohmann::json
   json input = "{ \"happy\": \"yes\", \"pi\": 3.141, \"t\": 1608008400000 }"_json;

   for(auto& el : input.items()) { 
       if (mp.find(el.key()) == mp.end()) {
           std::cout << "trying insertion for key " << el.key() << std::endl;
           mp.insert({ el.key(), std::vector>{ el.value() } });
           std::cout << "inserted " << el.key() << " successfully!" << std::endl;
       }
    }
    return 0;
}

Here is my input (note that my input is being passed to this program as type nlohmann::json): {"c":127.88,"h":127.9,"l":124.13,"n":867462,"o":124.34,"t":1608008400000,"v":157572262.0,"vw":126.5535},{"c":127.81,"h":128.37,"l":126.56,"n":550012,"o":127.41,"t":1608094800000,"v":95913591.0,"vw":127.5459}

And here is the current output:

And here is the current output

inserted c successfully!
trying insertion for key h
inserted h successfully!
trying insertion for key l
inserted l successfully!
trying insertion for key n
inserted n successfully!
trying insertion for key o
inserted o successfully!
trying insertion for key t
[1]    40305 killed     ./test

I have tried to address a bunch of potential different issues regarding why this could be happening, but I have essentially confirmed via trial and error that the issue occurs when I try and use the std::variant for my vector type. When I assign all of the vectors a uniform type (double, for example), all insertions work perfectly. However, the issue is one of extensibility. Even though this example only contains doubles and longs, in the future I'd ideally like to be able to parse some data that looks like this: {"a": "test", "b": 1243.343, "c": 120910394023332} without error and have the return values be (with types shown for clarity):

a : vector{"test"}, b : vector{1243.343}, c : vector{120910394023332}

If there's anything that I can clarify that would help answer this, please just let me know and I'll add it in.

user_3943312 · Answer

The following vector declaration:

std::vector v{4};

This creates a vector default-initialized with 4 values. This overloaded std::vector constructor takes a single parameter that gives the initial size of the vector.

std::vector>{ el.value() } }

Keeping my introduction in mind, it should now be obvious that this is going to invoke the same constructor.

When I stepped through the above code in the debugger, my debugger revealed the fact that things quickly became unglued when 1608008400000 got passed into the constructor. The chances of my computer successfully creating a vector with one trillion, six hundred eight billion, eight million, and four hundred thousand values, of anything, are very, very slim.

el.value() does not return a variant. It returns a JSON value, and there's no ready-made mechanism for converting it to a variant. You have to do all the work yourself, something like:

   auto v=el.value();

   if (v.is_number())
   {
       if (v.is_number_float())
       {
           vv.emplace_back( (double)v);
       }
       else
       {
           vv.emplace_back( (long)v);
       }
   }
   else
   {
       vv.emplace_back( (std::string) v);
   }

   mp.insert({ el.key(), vv});

user_13464284 · Answer

I've managed to answer my own question (albeit in a roundabout way). I can check whether or not the type is integer with the is_number_integer() function in nlohmann::json. From there, if it is an integer, I can get the uint64_t value from it and then insert it into the variant vector. The only modification I had to make was to alter the variant vector to change the types available by removing long and adding uint64_t. Here's the new code:

int main() {
// map stores col name as string and vector with col type
    std::unordered_map>> mp;

   // input is type nlohmann::json
   // input.push_back(json::object_t::value_type("t", 1608008400000));
   json input = "{ \"happy\": true, \"pi\": 3.141, \"t\": 1608008400000 }"_json;


   for(auto& el : input.items()) {
       if (mp.find(el.key()) == mp.end()) {
            std::cout << "trying insertion for key " << el.key() << std::endl;
            std::cout << "value is " << el.value() << " with type " << el.value().type_name() << std::endl;
            if (el.value().is_number_integer()) {
                mp.insert({ el.key(), std::vector>{ el.value().get() } });
            }
            else {
                mp.insert({ el.key(), std::vector>{ el.value() } });
            }
            std::cout << "inserted " << el.key() << " successfully!" << std::endl;
       }
    }
    return 0;
}

Insert into unordered_map hanging on runtime

2 Answers