understanding php curl_multi_exec

Question 1

I'm trying to understand curl_multi_exec. I've copied a piece of the manual example here. So I'm wondering, how does it work? The first loop sends the http request I guess? But it then it is followed by a loop inside a loop using functions with seemingly undocumented flags..

I would like to download +=70 urls +=in parallel.

http://www.php.net/manual/en/function.curl-multi-exec.php

<?php
...
$active = null;
//execute the handles
do {
    $mrc = curl_multi_exec($mh, $active);
} while ($mrc == CURLM_CALL_MULTI_PERFORM);

while ($active && $mrc == CURLM_OK) {
    if (curl_multi_select($mh) != -1) {
        do {
            $mrc = curl_multi_exec($mh, $active);
        } while ($mrc == CURLM_CALL_MULTI_PERFORM);
    }
}
...
?>

Question 2

You can explore two article that describes this example.

PHP and curl_multi_exec

First, here's the high level. There are two outer loops. The first one is responsible for clearing out the curl buffer right now. The second one is responsible for waiting for more information, and then getting that information. This is an example of what is called blocking I/O. We block execution of the rest of the program until the network I/O is done. While this isn't the most preferable way in general to handle network I/O, it's really our only choice in single-threaded, synchronous PHP.

Doing curl_multi_exec the right way

First the $mrc variable and from the manual we learn that the response is a cURL code defined in the cURL Predefined Constants. In essence it is a regular response and as with any other PHP function curl_multi_exec is no different and only returns a response once it is finished. Which means there should be only ONE response. In a perfect world this single response is 0 (zero) or equal to the predefined constant CURLM_OK.

Question 3

I think this article explains it pretty well:

Type-coercing comparison operators will convert numeric strings to numbers

Just to quote the main issue here:

According to php language.operators.comparison, the type-coercing comparison operators will coerce both operands to floats if they both look like numbers, even if they are both already strings:

where both strings are using exponential notation, hence are treated as numeric strings, making loose comparison (==), coerce these strings to floats before actually "loosely" comparing them.

As a best practice and to prevent unexpected behaviour, always try to use identity equality (===), especially when dealing with strings.

Question 4

I Always interpreted it like:

Fatal error: Out of memory ([currently] allocated 32016932) (tried to allocate [additional] 25152 bytes)

But good Question if there is a bulletproof explanation.

Question 5

In your while loop, you need to do the following for each URL:

create a curl resource by using curl_init()
set options for resource by curl_setopt(..)

Then you need to create a multiple curl handle by using curl_multi_init() and adding all the previous individual curl resources by using curl_multi_add_handle(...)

Then finally you can do curl_multi_exec(...).

A good example can be found here: http://us.php.net/manual/en/function.curl-multi-exec.php

Question 6

I got it solved-Its the returned Order thats important to understand and combine with the result. For someone who may be looking for the answer:

      $curls = $listofurls;
      $curl_arr = array();
      $master = curl_multi_init();

      for($i = 0; $i < $node_count; $i++) {

        $curl_arr[$i] = curl_init($curls[$i][0]);
        curl_setopt($curl_arr[$i],CURLOPT_FRESH_CONNECT,true);
        curl_setopt($curl_arr[$i],CURLOPT_CONNECTTIMEOUT,10);
        curl_setopt($curl_arr[$i],CURLOPT_HEADER,true);
        curl_setopt($curl_arr[$i],CURLOPT_CUSTOMREQUEST,'HEAD');
        curl_setopt($curl_arr[$i],CURLOPT_RETURNTRANSFER,true);
        curl_setopt($curl_arr[$i],CURLOPT_NOBODY,true);
        curl_setopt($curl_arr[$i],CURLOPT_AUTOREFERER, 1);
        curl_setopt($curl_arr[$i],CURLOPT_TIMEOUT,30);

        curl_multi_add_handle($master, $curl_arr[$i]);
      }



          $finalresult = array();
          $returnedOrder = array();

          do{

            curl_multi_exec($master, $running);
            $info = curl_multi_info_read($master);

            if($info['handle']) {
              $finalresult[] = curl_multi_getcontent($info['handle']);
              $returnedOrder[] = array_search($info['handle'], $curl_arr, true);
              curl_multi_remove_handle($master, $info['handle']);
              curl_close($curl_arr[end($returnedOrder)]);
            }

          $previousActive = $running;
          }
          while($running > 0);

          $res = array_combine($returnedOrder, $finalresult);
          curl_multi_close($master);

understanding php curl_multi_exec

Answers

php,curl,curl-multi