CURL - massive Performance issue

[aeberhard]

Description

The following code:

<?php

$starttime = microtime(true);

$ch = curl_init();
if (0 === curl_errno($ch)) {
    $header = [];
    $header[] = 'Accept: application/vnd.github+json';
    $header[] = 'X-GitHub-Api-Version:2022-11-28';

    $options = [
        CURLOPT_URL => 'https://api.github.com/rate_limit',
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_USERAGENT => 'PHP',
        //CURLOPT_CONNECTTIMEOUT => 0,
        //CURLOPT_TIMEOUT => 5,
        //CURLOPT_HTTPHEADER => $header,
        //CURLOPT_IPRESOLVE => CURL_IPRESOLVE_V4,
        //CURLOPT_ENCODING => '',
        //CURLOPT_DNS_USE_GLOBAL_CACHE => false,
    ];

    $log = fopen(__DIR__. '/curl.log', 'w');
    if (false !== $log) {
        $logoptions = [
            CURLOPT_VERBOSE => true,
            CURLOPT_STDERR => $log,
        ];
    }

    $curloptions = $options + $logoptions;
    curl_setopt_array($ch, $curloptions);

    $response = curl_exec($ch);
    $info = curl_getinfo($ch);

    $httpCode = (string) curl_getinfo($ch, CURLINFO_HTTP_CODE);
    echo '<pre>';
    var_dump([$httpCode, $response, $info]);
    echo '</pre>';
}


$duration = microtime(true) - $starttime;
echo "Runtime: $duration Sek.";

As you can see I tried with several Curl-Options

Resulted in this output:

Runtime: 0.76369309425354 Sek.
Runtime: 0.6199951171875 Sek.
Runtime: 0.6086049079895 Sek.
Runtime: 0.56775498390198 Sek.
Runtime: 0.54858207702637 Sek.

Performance of the Script in other PHP Versions on same Host:

PHP 7.4
Runtime: 0.11398887634277 Sek.
Runtime: 0.22311997413635 Sek.
Runtime: 0.1849901676178 Sek.
Runtime: 0.12297105789185 Sek.
Runtime: 0.16944289207458 Sek.

PHP 8.0
Runtime: 0.1490170955658 Sek.
Runtime: 0.11332702636719 Sek.
Runtime: 0.16415405273438 Sek.
Runtime: 0.19613790512085 Sek.
Runtime: 0.14559698104858 Sek.

PHP 8.1
Runtime: 0.13851690292358 Sek.
Runtime: 0.11393594741821 Sek.
Runtime: 0.10096192359924 Sek.
Runtime: 0.14624786376953 Sek
Runtime: 0.11632513999939 Sek.

In PHP 8.2 the the Curl pretransfer_time is up to 5 times bigger than in other versions.

PHP 8.0
    ["total_time"]=>
    float(0.127489)
    ["namelookup_time"]=>
    float(0.005659)
    ["connect_time"]=>
    float(0.058811)
    ["pretransfer_time"]=>
    float(0.105881)

PHP 8.2
    ["total_time"]=>
    float(0.522058)
    ["namelookup_time"]=>
    float(0.006427)
    ["connect_time"]=>
    float(0.036172)
    ["pretransfer_time"]=>
    float(0.5042)

PHP Version

PHP 8.2.0

Operating System

Windows 10 Enterprise 21H2

[aeberhard]

Curl Version 7.85.0

PHP Versions downloaded from https://windows.php.net/download/

[cmb69]

In PHP 8.2 the the Curl pretransfer_time is up to 5 times bigger than in other versions.

Indeed! It doesn't seem to be related to HTTP/2, since adding CURLOPT_SSL_ENABLE_ALPN => false shows the same performance degradation. Given that apparently the appconnect_time has degredated (please confirm or deny), this is likely an issue with OpenSSL 3 (which is used on Windows as of PHP 8.2.0), so this might not be Windows specific.

[aeberhard]

I confirm!
appconnect_time_us on PHP 8.2 is up to 5 times bigger!

PHP 8.0
    ["appconnect_time_us"]=>
    int(73811)
    ["connect_time_us"]=>
    int(33838)
    ["namelookup_time_us"]=>
    int(1862)
    ["pretransfer_time_us"]=>
    int(73967)

PHP 8.2
    ["appconnect_time_us"]=>
    int(369953)
    ["connect_time_us"]=>
    int(43670)
    ["namelookup_time_us"]=>
    int(16192)
    ["pretransfer_time_us"]=>
    int(370145)

[mwcng]

I experienced the same issue with curl requests becoming extremely sluggish after updating some of the software I was using, but after some testing it doesn't seem to be related to OpenSSL, PHP 8.2 or Windows.

I'm running Ubuntu 22.04 and I tried multiple OpenSSL versions: 3.0.5, 3.0.7 and the master branch version from today (3.2.0-dev), tried multiple PHP versions (8.1.9, 8.1.14 and 8.2.1) but no matter what I tried I couldn't seem to get curl to work fast like it did before with any of them. The problem ended up being curl itself. While updating my system I also installed the latest version of curl (7.87.0) and after reverting back to the previous version I had (7.84.0) curl performance in php went back to normal.

The issue was definitely related to https requests (although I didn't bother checking if it was http/1.1 or http/2 specific) considering that regular http requests were working fine even with the newer curl version. Considering that your problem started with curl 7.85.0 I guess we can assume that the problem starts from that version.

Right now I'm using PHP 8.2.1, OpenSSL 3.0.7 and cURL 7.84.0 and everything is ok, so for the time being if anyone is experiencing these problems you should probably just stick to using cURL 7.84.0 and update at a later point when this issue gets resolved.

[cmb69]

I just tried PHP 8.2 with libcurl 7.84.0 (Windows), and can see the same performance regression as with libcurl 7.85.0.

[mwcng]

I guess I assumed a bit too much :) I suppose the problem is a bit different on Windows but the performance regression is definitely not limited to Windows only.

[exactmultiple3425]

We experienced the same problem when using the official Docker image and upgrading from php:7.4-fpm to php:8.1-fpm. It was discovered that the problem started with a version of curl between 7.74 and 7.88. Downgrading to use the bullseye image with php:8.1 solved the issue.

[repli2dev]

We stumbled upon the same issue when migrating from PHP 8 to PHP 8.2 (using alpine official PHP images...) the version 8.2.8 (with curl 8.12) was OK, yet the issue seems to occasionally go disappear and reappear:

• 8.0.17 (uses alpine 3.15 and curl 7.80) does not have this issue.
• 8.0.8 (uses alpine 3.14 and curl 7.77) have this issue.

So probably just fiddle with the PHP + CURL versions...

Anyway it is testable using the above snippet and official docker images.