Code Coverage data serialized from PHPUnit PHAR cannot be processed by PHPCOV

[mstilkerich]

Q	A
PHPUnit version	9.5.0
PHP version	7.4.3
Installation Method	PHAR

Summary

When using PHPUnit phar archive (installed via phive), the generated php coverage report cannot be processed by phpcov 8.2.0 (also PHAR installed via phive). When using the same PHPUnit version installed via composer, phpcov processes the report.

Current behavior

• Execute PHP unit tests; have PHP unit create coverage in PHP format
• Execute phpcov to convert the .cov file produced by PHPUnit to clover

Result:

$ tools/phpunit -c tests/phpunit.xml
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.4.3 with Xdebug 2.9.2
Configuration: tests/phpunit.xml

.                                                                   1 / 1 (100%)

Time: 00:01.324, Memory: 584.00 MB

OK (1 test, 1 assertion)

Generating code coverage report in PHP format ... done [00:00]
$ tools/phpcov merge --clover testreports/unit/clover.xml testreports/unit
phpcov 8.2.0 by Sebastian Bergmann.

Failed to merge: testreports/unit/coverage.cov

When doing the same using a PHPunit installed via composer, everything works just fine:

$ vendor/bin/phpunit -c tests/phpunit.xml
PHPUnit 9.5.0 by Sebastian Bergmann and contributors.

Runtime:       PHP 7.4.3 with Xdebug 2.9.2
Configuration: tests/phpunit.xml

.                                                                   1 / 1 (100%)

Time: 00:00.629, Memory: 310.00 MB

OK (1 test, 1 assertion)

Generating code coverage report in PHP format ... done [00:00]
$ tools/phpcov merge --clover testreports/unit/clover.xml testreports/unit
phpcov 8.2.0 by Sebastian Bergmann.

Generating code coverage report in Clover XML format ... done

How to reproduce

Use the minimal example attached. The testreport directory also contains the output files that both the PHAR version as well as the composer version produced.

Expected behavior

PHAR version should work the same as the version from composer.

[mstilkerich]

testproj.zip

[cavo789]

Hi all.
For information, I've the same problem with Phpunit 9.5.3 and PhpCov 8.2.0.
The .cov file generated by phpunit.phar can't be processed by phpcov.phar ("Failed to merge...").
Exactly the same versions but this time using composer (not the binary) works.
There is something about the binaries for this.

Thanks Michael to have reported this and give me the direction for solving my problem.

[twoseascharlie]

@sebastianbergmann I have just ran into this same issue.

Using phpunit from direct wget download:
(wget -O phpunit https://phar.phpunit.de/phpunit-9.phar)

Then use phpunit to generate a few .cov files for a few different unit tests.
Then try to merge them with phpcov and just get "Failed to merge:" messages for all the .cov files.

If I use the phpunit from composer to run tests and generate .cov files, it works as expected as others here have commented.

I'm pretty sure I have figured out the reason why.

Here is my analysis:
I extracted the phpunit.phar downloaded from wget. When I view the file:
.../phpunitextracted/php-code-coverage/CodeCoverage.php

The declared namespace is:
namespace PHPUnit\SebastianBergmann\CodeCoverage;

When I view the same file from the composer install:
.../vendor/phpunit/php-code-coverage/src/CodeCoverage.php

The declared namespace is:
namespace SebastianBergmann\CodeCoverage;

Note the missing PHPUnit prefix.

I've also extracted the phpcov.phar. When I view the same file again:
.../phpcovextracted/phpunit/php-code-coverage/src/CodeCoverage.php

The declared namespace is:
namespace SebastianBergmann\CodeCoverage;

Also missing the PHPUnit prefix.

So, when the phpcov utility tries to unserialize the CodeCoverage object (MergeComand.php line 57), it does an instanceof check:

            if (!$_coverage instanceof CodeCoverage) {
                $errors[] = $file;

                unset($_coverage);

                continue;
            }

When using the phpunit.phar from wget to generate .cov files of CodeCoverage objects, they are serialized down with a different fully qualified class name than when using the composer tools. Since the fully qualified class names aren't the same, the check fails and an error is raised.

[sebastianbergmann]

Thank you, @twoseascharlie, that makes sense. We need to either "rewrite" the serialized object representation before we call unserialize() or do not use serialize() / unserialize() in the first place.

[kassner]

My .cov files had the PHPUnit\ prefix in the serialized classes, i.e.:

<?php
return \unserialize('O:51:"PHPUnit\SebastianBergmann\CodeCoverage\CodeCoverage":15:{s:59:" PHPUnit\SebastianBergmann\CodeCoverage\CodeCoverage driver";....

But the CodeCoverage classes are under a different namespace.

I've used the script below to convert the files:

<?php

$content = file_get_contents($_SERVER['argv'][1]);

echo preg_replace_callback("#\:(\d+)\:\"(.?)PHPUnit\\\SebastianBergmann\\\#", function($matches) {
    $hasNull = strlen($matches[2]) === 1;
    $size = $matches[1];
    $size -= 8;

    if ($hasNull) {
        return sprintf(":%d:\"\0SebastianBergmann\\", $size);
    }

    return sprintf(':%d:"SebastianBergmann\\', $size);
}, $content);

I managed to merge a couple of .cov files together after this. I've closed phpcov + composer install to run it, have not tried with the PHAR.

It's a workaround nonetheless, I wouldn't add this to my CI/CD pipeline. Use at your own risk.

[ju1ius]

FWIW, it still happens with phpunit:9.5.20 and phpcov:8.2.1

<!-- .phive/phars.xml -->
<?xml version="1.0" encoding="UTF-8"?>
<phive xmlns="https://phar.io/phive">
  <phar name="phpunit" version="^9.5.20" installed="9.5.20" location="./tools/phpunit.phar" copy="true"/>
  <phar name="phpcov" version="^8.2.1" installed="8.2.1" location="./tools/phpcov.phar" copy="true"/>
</phive>

[NicoHaase]

@sebastianbergmann do you know of any way to circumvent this behaviour? Even after applying the workaround provided by kassner, phpcov throws the following error message:

PHP Fatal error:  Uncaught TypeError: Cannot assign __PHP_Incomplete_Class to property SebastianBergmann\CodeCoverage\CodeCoverage::$driver of type SebastianBergmann\CodeCoverage\Driver\Driver in /var/www/html/project/phpunit-coverage.cov:3
Stack trace:
#0 /var/www/html/project/phpunit-coverage.cov(3): unserialize('O:43:"Sebastian...')
#1 phar:///var/www/html/project/bin/phpcov.phar/src/cli/MergeCommand.php(55): include('/var/www/html/p...')
#2 phar:///var/www/html/project/bin/phpcov.phar/src/cli/Application.php(48): SebastianBergmann\PHPCOV\MergeCommand->run(Object(SebastianBergmann\PHPCOV\Arguments))
#3 /var/www/html/project/bin/phpcov.phar(1320): SebastianBergmann\PHPCOV\Application->run(Array)
#4 {main}
  thrown in /var/www/html/project/phpunit-coverage.cov on line 3

FTR: I'm testing this with phpunit.phar v9.6.8 and phpcov.phar v9.0.0

[afflerbach]

@kassner's great workaround from above in a single line (Perl, though 😉):

$ perl -i -pe 's#:(\d+):"(\x0?)PHPUnit\\(SebastianBergmann\\)#":".($1-8).":\"$2$3"#ge;' foo.cov

• PHPUnit 10.2.3
• PHP 8.2.7 with PCOV 1.0.11
• phpcov 9.0.0

[sebastianbergmann]

• Data generated using Composer-installed PHPUnit can be processed by Composer-installed PHPCOV
• Data generated using Composer-installed PHPUnit can be processed by PHPCOV's PHAR
• Data generated using PHPUnit's PHAR cannot be processed by Composer-installed PHPCOV
• Data generated using PHPUnit's PHAR cannot be processed by PHPCOV's PHAR

Do I understand the current situation correctly?

If that is the case, then a short-term solution could be to introduce PHP-Scoper into the build process of PHPCOV's PHAR. Then data generated using PHPUnit's PHAR should be processable by PHPCOV's PHAR.

We could also look into using class_alias() to alias SebastianBergmann\CodeCoverage (unscoped) and PHPUnit\SebastianBergmann\CodeCoverage (scoped).

I hope to find the time it takes to "dust off" PHPCOV and fix all current issues soon.