Perform ~500 queries from PHP using all the modern features of HTTP/2

Viewed 223

I develop some middleware (actually scraping tool) and need send ~500 POST requests to one server for receiving answers. Server supports HTTP/2. I'm using PHP 7.1 with libcurl/7.53.1, i.e. all versions are suitable.

How to fastly perform ~500 queries from PHP using all the modern features of HTTP/2? I wrote a lot of scripts with the usual reusing curl handle but really not understand what needed for analogous handling responses with multiplexing.

My code:

$mh = curl_multi_init();
$chs = array();
$active = 0;
curl_multi_setopt($mh, CURLMOPT_PIPELINING, 1);
curl_multi_setopt($mh, CURLMOPT_MAXCONNECTS, 8);

$res = array();
$portals = array();

for ($i=0; $i<count($tiles); $i++) {
    $post = json_encode($tile_data);
    $ch = curl_init();
    $options = array(
        CURLOPT_URL => "https://www.example.com/v1/getTile",
        CURLOPT_POST => true,
        CURLOPT_POSTFIELDS => $post,
        CURLOPT_RETURNTRANSFER => 1,
        CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_2_0
    );
    curl_setopt_array($ch, $options);
    $chs[$tiles[$i]] = $ch;
    curl_multi_add_handle($mh, $ch);
}

do {
    $mrc = curl_multi_exec($mh, $active);
} while ($active > 0);

for ($i=0; $i<count($tiles); $i++) {
    $ch = $chs[$tiles[$i]];
#    $req_ok = false;
#    while ($req_ok == false) {
        $raw = curl_multi_getcontent($ch);
        $data = json_decode($raw, true);
#        if (isset($data['result']['map'][$tiles[$i]]['entities'])) {
#            $req_ok = true;
            curl_multi_remove_handle($mh, $ch);
#        }
#    }
    array_push($res, $data);
}

It works, but if I uncomment strings for checking response, it no works more (hanging up). I need to check answers and resend request if server returned something wrong. Also I unaware, where in this code I can put function, which will be renders progressbar to my console, because I'm not clear, when separate requests are sent. I hope anyone already realized similar code and can explain me what I need to change. Thanks.

0 Answers
Related