proc_open mixed up stderr and stdout when using pty for pipe

[joelwurtz]

Description

We were having randomly bug in our ci where sometimes some test would failed because the stdout of a sub process launched by php was not complete.

We manage to make a reproducer, and it seems that sometimes stdout output would go to stderr.

Here is a reproducer :

<?php

[$reference] = run_process();

echo "Output Reference:\n";
echo $reference;
echo "\n";

for ($i=0; $i < 10_000; $i++) {
    if ($i > 0 && $i % 80 === 0) {
        echo "\n";
    }
    echo '.';
    [$output, $error] = run_process();

    // uncomment this line to make it works
//    $output .= $error;

    
    if ($output !== $reference) {
        throw new \RuntimeException("Output mismatch");
    }
}

function run_process(){
    $descriptors = [
        ['pty'],
        ['pty'],
        ['pty'],
    ];
    $pipes = [];

    $proc = proc_open(['cat', __FILE__], $descriptors, $pipes);
    $output = "";
    $error = "";

    stream_set_blocking($pipes[1], 0);
    stream_set_blocking($pipes[2], 0);

    while (proc_get_status($proc)['running']) {
        usleep(10); // longer timer will not cause the issue

        $output .= @stream_get_contents($pipes[1], -1);
        $error .= @stream_get_contents($pipes[2], -1);
    }

    $output .= @stream_get_contents($pipes[1], -1);
    $error .= @stream_get_contents($pipes[2], -1);

    return [$output, $error];
}

I suspect this have to do about having a pipe not ready or something mixed, having a longer timer before first read, reduce a lot the probability of error

See also symfony/symfony#59927

PHP Version

PHP 8.4

Operating System

No response

[ktomk]

We manage to make a reproducer, and it seems that sometimes stdout output would go to stderr.

Yes, it's a reproducer, but take a look what it reproduces:

$descriptors = [
    ['pty'],
    ['pty'],
    ['pty'],
];

IIRC, PHP is using a single PTY character device and then what you experience is pretty explicable:

    $output .= @stream_get_contents($pipes[1], -1);
    $error .= @stream_get_contents($pipes[2], -1);

You are reading from the same device, but you put it on different variables. If data arrives between those two calls, it will be appended to $error --not $output.

[joelwurtz]

It's the same pty but different file descriptors (when looking at the strace) so i don't exepct this behavior ? and there is no way to use a different pty (and i don't think it makes sense)

Same behavior is observed when using the stream_select function (that's what it use in symfony and from where we observed this behavior)

[joelwurtz]

However i just try to open an explicit pty for each pipe and this error is not preset anymore, will throw a PR to see if it's accepted

[wez]

FWIW, a PTY has just an input stream and an output stream. There is no separation of stdout or stderr once the data has been sent to the PTY. If you spawn a program into a PTY, its stdout and stderr are emitted to the PTY and blended there, at that interface to the kernel, when it writes to the PTY.

When you read from a PTY you are reading its single output stream.

[joelwurtz]

Clothing this as "not a bug"