Skip to content

Commit 903ddbe

Browse files
committed
minor #15491 [Contribution] Document the different types of tests run in CI (wouterj)
This PR was merged into the 4.4 branch. Discussion ---------- [Contribution] Document the different types of tests run in CI The goal of this PR is two sided: 1. Help contributors find out why a specific test job is failing by explaining which scenarios are tested by which job name 2. Reduce the bus factor of the `symfony/symfony` workflows `@nicolas`-grekas I would love if you could have a look if I'm not talking nonsense here 😄 Commits ------- 28e7cbe [Contribution] Document the different types of tests run in CI
2 parents 09823c2 + 28e7cbe commit 903ddbe

File tree

1 file changed

+62
-0
lines changed

1 file changed

+62
-0
lines changed

contributing/code/pull_requests.rst

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -429,6 +429,68 @@ After the `Psalm phar is installed`_, the analysis can be run locally with:
429429
430430
$ psalm.phar src/Symfony/Component/Workflow
431431
432+
Automated Tests
433+
~~~~~~~~~~~~~~~
434+
435+
A series of automated tests will run when submitting the pull request.
436+
These test the code under different conditions, to be sure nothing
437+
important is broken. Test failures can be unrelated to your changes. If you
438+
think this is the case, you can check if the target branch has the same
439+
errors and leave a comment on your PR.
440+
441+
Otherwise, the test failure might be caused by your changes. The following
442+
test scenarios run on each change:
443+
444+
``PHPUnit / Tests``
445+
This job runs on Ubuntu using multiple PHP versions (each in their
446+
own job). These jobs run the testsuite just like you would do locally.
447+
448+
A failure in these jobs often indicates a bug in the code.
449+
450+
``PHPUnit / Tests (high-deps)``
451+
This job checks each package (bridge, bundle or component) in ``src/``
452+
individually by calling ``composer update`` and ``phpunit`` from inside
453+
each package.
454+
455+
A failure in this job often indicates a missing package in the
456+
``composer.json`` of the failing package (e.g.
457+
``src/Symfony/Bundle/FrameworkBundle/composer.json``).
458+
459+
This job also runs relevant packages using a "flipped" test (indicated
460+
by a ``^`` suffix in the package name). These tests checkout the
461+
previous major release (e.g. ``4.4`` for a pull requests on ``5.4``)
462+
and run the tests with your branch as dependency.
463+
464+
A failure in these flipped tests indicate a backwards compatibility
465+
break in your changes.
466+
467+
``PHPUnit / Tests (low-deps)``
468+
This job also checks each package individually, but then uses
469+
``composer update --prefer-lowest`` before running the tests.
470+
471+
A failure in this job often indicates a wrong version range or a
472+
missing package in the ``composer.json`` of the failing package.
473+
474+
``continuous-integration/appveyor/pr``
475+
This job runs on Windows using the x86 architecture and the lowest
476+
supported PHP version. All tests first run without extra PHP
477+
extensions. Then, all skipped tests are run using all required PHP
478+
extensions.
479+
480+
A failure in this job often indicate that your changes do not support
481+
Windows, x86 or PHP with minimal extensions.
482+
483+
``Integration / Tests``
484+
Integration tests require other services (e.g. Redis or RabbitMQ) to
485+
run. This job only runs the tests in the ``integration`` PHPUnit group.
486+
487+
A failure in this job indicates a bug in the communication with these
488+
services.
489+
490+
``PHPUnit / Tests (experimental)``
491+
This job always passes (even with failing tests) and is used by the
492+
core team to prepare for the upcoming PHP versions.
493+
432494
.. _rework-your-patch:
433495

434496
Rework your Pull Request

0 commit comments

Comments
 (0)