Cobertura XML and GitLab CI

[sebastianbergmann]

Considering https://gitlab.com/gitlab-org/gitlab/-/issues/296012 and #734 (comment), would it make sense to (optionally) put the complete path into the "filename" attribute?

[realFlowControl]

Hey there,

I wanted to reproduce the bug but couldn't. Looks like GitLab actually fixed this with https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html#automatic-class-path-correction in GitLab 13.9 earlier this year. I pinged Colin O'Dell in the GitLab issue if he is still able to reproduce the bug.
At this point in time it looks like there is no change needed in PHPUnit Cobertura XML report.

I can ping you here when I know more.

/Flo

[sebastianbergmann]

Thank you, Florian. Do ping me here when you know more.

[dhirtzbruch]

Hi there,

I know this is a quite old issue, however I just tried to get the cobertura report running with my gitlab instance.

While the report was correctly generated, gitlab did not show the line coverage in merge requests using the following phpunit configuration. I think the automatic class path correction did not work as expected, so I changed to use full relative paths.

    <coverage>
        <include>
            <directory>./src/</directory>
        </include>
        <report>
            <cobertura outputFile="build/cobertura.xml"/>
        </report>
    </coverage>

However, changing the config to the following (shortened) worked for me - however I did have to add all non-code-directories to the exclusion list:

    <coverage>
        <include>
            <directory>./</directory>
        </include>
        <exclude>
            <directory>bin/</directory>
            <directory>tests/</directory>
            <directory>var/</directory>
            <directory>vendor/</directory>
        </exclude>
        <report>
            <text outputFile="php://stdout"/>
            <cobertura outputFile="build/cobertura.xml"/>
        </report>
    </coverage>

Just in case anyone views this issue having the same problem.

Would it be possible to maybe add a configuration option to the cobertura element like pathsRelativeTo or so @sebastianbergmann? Like this:

        <report>
            <cobertura outputFile="build/cobertura.xml" pathsRelativeTo="./"/> 
        </report>

I'd love to try if I can create a merge request - just wanted to ensure the idea is ok and there's no mistake on my side.

[sebastianbergmann]

It "just works" as @realFlowControl explains in https://dev.to/realflowcontrol/revisiting-gitlab-as-a-php-developer-5h9l.

[Jean85]

@sebastianbergmann unfortunately I just hit this same bug, because I run my tests with a "Docker in Docker" approach, so the absolute path does not match what GitLab expects, hence stopping the automatic class path correction to help me.

Can we reopen this issue?

[EDIT] Also, I must add that the workaround in #881 (comment) is not working for me, the Cobertura coverage is still generated with the src/ prefix path under the sources node, and missing from the filename attribute, so the path does not match even in that case.

[sebastianbergmann]

@Jean85 Sure, we can reopen this issue. I cannot promise to work on this myself anytime soon, but I would review/merge any PR ASAP.

[Jean85]

@dhirtzbruch can you tell us how that workaround worked for you, or with which package versions? I rolled back to PHPUnit 9.4.0 and php-code-coverage 9.2.0 (the first versions to introduce cobertura support) and I still get the wrong path :(

[Jean85]

I think I found the issue by debugging it: it's not in the Cobertura implementation but one step before, in \SebastianBergmann\CodeCoverage\Node\Builder::reducePaths: since all my covered code is inside src/, that method is used to build a report with the highest level of common path, hence breaking the path correspondence between GitLab (which expects the path starting from the root of the project/repo) and the produced source node in the Cobertura XML output file.

So, including some covered file outside src/ would be a functional workaround for me... @sebastianbergmann what should I do? Can I fix this some other way?

[sebastianbergmann]

reducePaths() returns the common path which probably needs to be prefixed to the paths for Cobertura XML. I do not know how much effort it would be to refactor the existing code to make that information available where it is needed.

[dkarlovi]

In Gitlab docs they say:

Automatic class path correction only works on source paths in the format <CI_BUILDS_DIR>/<PROJECT_FULL_PATH>/.... The source is ignored if the path does not follow this pattern. The parser assumes that the filename of a class element contains the full path relative to the project root.

I had a debugging session where I create the coverage data in a Docker container which thinks the project is mounted at /project which then gets put in the report, this is fine and my local issue, but then it seems a better approach would be to make the file paths in the report relative to the project root to take advantage of this and make it more consistent.

Basically, instead of

<coverage>
  <sources>
    <source>/project/src</source>
  </sources>
  <packages>
    <package name="Foo.php">

do

<coverage>
  <sources>
    <source>/project</source>
  </sources>
  <packages>
    <package name="src/Foo.php">

and the problem should just go away, no?

[dkarlovi]

@Jean85 the solution I'll try is to force my Docker container volume to be the same as the host CWD. This should mean the sources path generated will match what Gitlab expects. It's way more complex for sure, but I don't see how else this could get fixed if this project doesn't want to fix it as described in the previous comment.

Basically, something like

docker run -v "./:`pwd`" -w "`pwd`" ...

Update: I've actually done this in sigwinhq/infra#168 and the coverage started showing up on Gitlab for me, from my POV this is solved.