Skip to content

Commit 08354ca

Browse files
authored
Update reporting.md
1 parent 0fe97bb commit 08354ca

File tree

1 file changed

+15
-22
lines changed

1 file changed

+15
-22
lines changed

docs/reporting.md

Lines changed: 15 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,7 @@ The `mftf` tool logs output continuously to the `dev/tests/acceptance/mftf.log`
1818

1919
## Command line
2020

21-
The MFTF reports about its progress during test run when you run the `mftf` CLI tool with [`run:test`][] or [`run:group`][] commands.
21+
MFTF reports about its progress during test run when you run the `mftf` CLI tool with [`run:test`][] or [`run:group`][] commands.
2222

2323
The report can contain three main parts:
2424

@@ -35,14 +35,7 @@ The following sections demonstrate an example interpretation of a complete log s
3535

3636
### Pre-run check report
3737

38-
First, the MFTF reports about issues with environment.
39-
In our case, there is an issue with PHP library loading.
40-
41-
```terminal
42-
PHP Warning: PHP Startup: Unable to load dynamic library '/usr/local/lib/php/pecl/20160303/php_intl.dll' - dlopen(/usr/local/lib/php/pecl/20160303/php_intl.dll, 9): image not found in Unknown on line 0
43-
```
44-
45-
Next, the MFTF returns `DEPRECATION` reports alerting you that required test components are missing in XML test definitions.
38+
First, MFTF returns `DEPRECATION` warnings alerting you that required test components are missing in XML test definitions.
4639

4740
```terminal
4841
DEPRECATION: Test AdminFilteringCategoryProductsUsingScopeSelectorTest is missing required annotations.{"testName":"AdminFilteringCategoryProductsUsingScopeSelectorTest","missingAnnotations":"stories"}
@@ -51,7 +44,7 @@ DEPRECATION: Test AdminRemoveProductWeeeAttributeOptionTest is missing required
5144
Generate Tests Command Run
5245
```
5346

54-
`Generate Tests Command Run` indicates the moment when the MFTF has run the tests generation command actually.
47+
`Generate Tests Command Run` indicates that test generation is finished and tests are able to be executed.
5548

5649
### Test execution report
5750

@@ -72,7 +65,7 @@ Magento\FunctionalTestingFramework.functional Tests (2) ------------------------
7265
Modules: \Magento\FunctionalTestingFramework\Module\MagentoWebDriver, \Magento\FunctionalTestingFramework\Helper\Acceptance, \Magento\FunctionalTestingFramework\Helper\MagentoFakerData, \Magento\FunctionalTestingFramework\Module\MagentoRestDriver, PhpBrowser, \Magento\FunctionalTestingFramework\Module\MagentoSequence, \Magento\FunctionalTes
7366
```
7467

75-
After the test generation command (mentioned in the previous section), the MFTF delegates control to the `vendor/codeception` tool, which is the `Codeception PHP Testing Framework` of version `2.3.9` that uses `PHPUnit` of version `6.5.13`.
68+
After the test generation command (mentioned in the previous section), MFTF delegates control to the `vendor/codeception` tool, which is the `Codeception PHP Testing Framework` of version `2.3.9` that uses `PHPUnit` of version `6.5.13`.
7669

7770
The tool runs `2 Tests` using the configuration defined in the `functional` suite under the `Magento\FunctionalTestingFramework` namespace.
7871
The corresponding configuration file is `acceptance/tests/functional.suite.yml`.
@@ -154,7 +147,7 @@ I save screenshot
154147
FAIL
155148
```
156149

157-
When a test step fails, the MFTF always saves a screenshot of the web page with the failing state immediately after the failure occurs.
150+
When a test step fails, MFTF always saves a screenshot of the web page with the failing state immediately after the failure occurs.
158151
`I save screenshot` follows the failing test step `I see "#something"` in our case.
159152

160153
A screenshot of the fail goes at the `acceptance/tests/_output` directory in both PNG and HTML formats:
@@ -176,7 +169,7 @@ Actions after `FAIL` are run as a part of the [`after`][] hook of the test.
176169

177170
### Test result report
178171

179-
After the MFTF completed test execution, it generates a general report about test results along with detailed information about each fail.
172+
After MFTF completed test execution, it generates a general report about test results along with detailed information about each fail.
180173

181174
```terminal
182175
--------------------------------------------------------------------------------
@@ -192,7 +185,7 @@ First you see warnings and deprecations.
192185
The `DEPRECATION` here is thrown by an MFTF dependency (Symfony) that is out of the scope for test writers and should be considered by MFTF contributors.
193186
If you encounter this type of reporting, [report an issue][].
194187

195-
Then, the MFTF reports that the test run took 52.43 seconds using 16 MB of system RAM.
188+
Then, MFTF reports that the test run took 52.43 seconds using 16 MB of system RAM.
196189
And, finally, that there was `1 failure`.
197190

198191
Next, the report provides details about the test failure.
@@ -268,19 +261,25 @@ FAILURES!
268261
Tests: 2, Assertions: 3, Failures: 1.
269262
```
270263

271-
The MFTF encountered failures due to the last test run, that included *2* tests with *3* assertions.
264+
MFTF encountered failures due to the last test run, that included *2* tests with *3* assertions.
272265
*1* assertion fails.
273266

274267
## Allure
275268

276-
Each time you run tests, the MFTF appends an XML file with results at the `tests/_output/allure-results/` directory.
269+
Each time you run tests, MFTF appends an XML file with results at the `tests/_output/allure-results/` directory.
277270

278271
The official [Allure Test Report][] documentation is well-covered, so we'll list only the CLI commands that you would need for your day-to-day work.
279272

280273
<div class="bs-callout bs-callout-info">
281274
The following commands are relative to the Magento installation directory.
282275
</div>
283276

277+
To generate the HTML Allure report in a temporary folder and open the report in your default web browser:
278+
279+
```bash
280+
allure serve dev/tests/acceptance/tests/_output/allure-results/
281+
```
282+
284283
To generate a report to the `allure-report/` at the current directory:
285284

286285
```bash
@@ -334,12 +333,6 @@ To clean up existing reports before generation (for example after getting new re
334333
allure generate dev/tests/acceptance/tests/_output/allure-result --clean
335334
```
336335

337-
To generate the HTML Allure report in a temporary folder and open the report in your default web browser:
338-
339-
```bash
340-
allure serve dev/tests/acceptance/tests/_output/allure-results/
341-
```
342-
343336
Refer to the [Reporting section][] for more Allure CLI details.
344337

345338
<!-- Link definitions -->

0 commit comments

Comments
 (0)