simdjson c++ - function arguments from known-length arrays

user_5861213 0
Viewed 270

I'm using the simdjson library to parse two lists of lists within a json object. The json data are provided by a third party, so I cannot modify the data structure.

The code looks like the following:

#include <iostream>
#include "simdjson.h"

using namespace simdjson;

class Game{
    public:
    void update_a(float a, float b){
        std::cout << "A: " << a << " - " << b << std::endl;
    };
    void update_b(float a, float b){
        std::cout << "B: " << a << " - " << b << std::endl;
    };
};

int main(void)
{
    Game game = Game();
    ondemand::parser parser;
    auto json = R"( {
      "data": {
        "player_a": [
            ["x", 0.4, 1.999],
            ["y", 0.99, 1.24],
            ["x", 0.2, 1.45],
            ["z", 0.41, 1.1],
        ],
        "player_b": [
            ["w", 1.34, 1.3],
            ["q", 1.34, 1.3],
            ["x", 1.34, 1.3],
        ]
      }
    } )"_padded;
    auto doc = parser.iterate(json);

    ondemand::object data;
    ondemand::array player_a;
    ondemand::array player_b;

    auto error = doc["data"].get(data);
    if (error) { return 1; }

    error = data["player_a"].get_array().get(player_a);
    if (error) {
        std::cout << "Player A error: " << error << std::endl;
        return 1;
    }
    for (auto move: player_a)
    {
        double up;
        double right;
        move.at(1).get(up);
        move.at(2).get(right);
        game.update_a(up, right);
    }

    error = data["player_b"].get_array().get(player_b);
    if (error) {
        std::cout << "Player B error: " << error << std::endl;
        return 1;
    }
    for (auto move: player_b)
    {   
        std::cout << move << std::endl;
        double up;
        double right;
        move.at(1).get(up);
        move.at(2).get(right);
        game.update_b(up, right);
    }
}

Basically, I don't know a priori the lenght of the player_a and player_b arrays, but I know that each element in those arrays are fixed lenght arrays (three elements) and I have to call a function that takes as arguments the second and the third elements of such arrays.

The code above generate the following output, which is wrong:

A: 0.4 - 0
A: 0.99 - 0
A: 0.2 - 0
A: 0.41 - 0
A: 0.41 - 0
A: 0.41 - 0
A: 0.41 - 0
Player B error: Objects and arrays can only be iterated when they are first encountered.

Is maybe the .at(1) method used wrongly?

How could a correct and performant code look like?

arrays parsing c++ json
1 Answers

Yup, there's a very important caveat in the documentation of at(): https://simdjson.org/api/1.0.0/classsimdjson_1_1_s_i_m_d_j_s_o_n___i_m_p_l_e_m_e_n_t_a_t_i_o_n_1_1ondemand_1_1array.html#a64b2a92de916b963eb298be7b2340f7c

This function should only be called once as the array iterator is not reset between each call.

So, yeah, you're using at(2) wrong, but it's not your fault. Naming the method at() would give anyone a real expectation that it will work exactly like at() on an std::vector. Given the library's narrow focus on speed, I wouldn't be surprised if at(N) is just skipping N commas.

Anyway, you have to reset the iterator before using at again. Unfortunately, the reset method isn't available on the base ondemand::value. So you have to cast move to an ondemand::array

for (auto move: player_a)
{
    ondemand::array move_array;
    error = move.get_array().get(move_array);
    if (error) {
        std::cout << "move error: " << error << std::endl;
        return 1;    
    }

    double up;
    double right;
    move_array.at(1).get(up);
    move_array.reset();
    move_array.at(2).get(right);
    game.update_a(up, right);
}

Alternatively, if you didn't want to backtrack, you could manually step through an iterator:

for (auto move: player_a)
{
    auto it = move.begin();
    ++it; // skip name

    double up;
    (*it).get(up);
    ++it;

    double right;
    (*it).get(right);
    game.update_a(up, right);
}

While testing this, I learned that the extra comma at the end of the JSON array causes it to do another iteration, and report a type error during move.get_array().get(...). So, make sure the JSON is well formed because the error message is confusing!

user_18639380
Related