I develop some middleware (actually scraping tool) and need send ~500 POST requests to one server for receiving answers. Server supports HTTP/2. I'm using PHP 7.1 with libcurl/7.53.1, i.e. all versions are suitable.
How to fastly perform ~500 queries from PHP using all the modern features of HTTP/2? I wrote a lot of scripts with the usual reusing curl handle but really not understand what needed for analogous handling responses with multiplexing.
My code:
$mh = curl_multi_init();
$chs = array();
$active = 0;
curl_multi_setopt($mh, CURLMOPT_PIPELINING, 1);
curl_multi_setopt($mh, CURLMOPT_MAXCONNECTS, 8);
$res = array();
$portals = array();
for ($i=0; $i<count($tiles); $i++) {
$post = json_encode($tile_data);
$ch = curl_init();
$options = array(
CURLOPT_URL => "https://www.example.com/v1/getTile",
CURLOPT_POST => true,
CURLOPT_POSTFIELDS => $post,
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_2_0
);
curl_setopt_array($ch, $options);
$chs[$tiles[$i]] = $ch;
curl_multi_add_handle($mh, $ch);
}
do {
$mrc = curl_multi_exec($mh, $active);
} while ($active > 0);
for ($i=0; $i<count($tiles); $i++) {
$ch = $chs[$tiles[$i]];
# $req_ok = false;
# while ($req_ok == false) {
$raw = curl_multi_getcontent($ch);
$data = json_decode($raw, true);
# if (isset($data['result']['map'][$tiles[$i]]['entities'])) {
# $req_ok = true;
curl_multi_remove_handle($mh, $ch);
# }
# }
array_push($res, $data);
}
It works, but if I uncomment strings for checking response, it no works more (hanging up). I need to check answers and resend request if server returned something wrong. Also I unaware, where in this code I can put function, which will be renders progressbar to my console, because I'm not clear, when separate requests are sent. I hope anyone already realized similar code and can explain me what I need to change. Thanks.