You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: CHANGELOG.md
+26Lines changed: 26 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,32 @@
1
1
Magento Functional Testing Framework Changelog
2
2
================================================
3
3
4
+
2.4.0
5
+
-----
6
+
### Enhancements
7
+
* Maintainability
8
+
* Added new `mftf static-checks` command to run new static checks against the attached test codebase
9
+
* See DevDocs for details
10
+
* Added new `mftf generate:docs` command that generates documentation about attached test codebase
11
+
* See DevDocs for details
12
+
* Traceability
13
+
* Allure reports for tests now contain collapsible sections for `actionGroup`s used in execution.
14
+
15
+
### Fixes
16
+
* Fixed an issue where `magentoCli` would treat `argument="0"` as a null value.
17
+
* Fixed an issue where `amOnPage` and `waitForPwaElementVisible` would not utilize the `timeout` attribute correctly when MagentoPwaWebDriver is enabled.
18
+
* Fixed an issue where invalid XML characters would cause Allure to throw an exception without a resulting report.
19
+
* Fixed `codeception.dist.yml` configuration for keeping previous test run results.
20
+
* PHP Notices are no longer thrown when XML is missing non-necessary attributes.
21
+
* Removed unusable `fillSecretField` action from schema.
22
+
23
+
### GitHub Issues/Pull requests:
24
+
*[#338](https://github.com/magento/magento2-functional-testing-framework/pull/338) -- Return exit codes of process started by 'run:test', 'run:group' or 'run:failed' command
25
+
*[#333](https://github.com/magento/magento2-functional-testing-framework/pull/333) -- Added Nginx specific settings to getting started doc
26
+
*[#332](https://github.com/magento/magento2-functional-testing-framework/pull/332) -- executeInSelenium action does not generate proper code
27
+
*[#318](https://github.com/magento/magento2-functional-testing-framework/pull/318) -- Reduce cyclomatic complexity in Problem Methods
28
+
*[#287](https://github.com/magento/magento2-functional-testing-framework/pull/287) -- Update requirements to include php7.3 support
| `--config=[<default>|<singleRun>|<parallel>]` | Creates a single manifest file with a list of all tests. The default location is `tests/functional/Magento/FunctionalTest/_generated/testManifest.txt`.<br/> You can split the list into multiple groups using `--config=parallel`; the groups will be generated in `_generated/groups/` like `_generated/groups/group1.txt, group2.txt, ...`.</br> Available values: `default` (default), `singleRun`(same as `default`), and `parallel`.</br> Example: `generate:tests --config=parallel`. |
121
-
|`--force`| Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`.|
122
-
|`-i,--time`| Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
123
-
|`--tests`| Defines the test configuration as a JSON string.|
124
-
|`--debug`| Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This parameter takes extra processing time. Use it after test generation has failed once.|
125
-
|`-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
119
+
| Option | Description|
120
+
| ---|--- |
121
+
|`--config=[<default> or <singleRun> or <parallel>]`| Creates a single manifest file with a list of all tests. The default location is `tests/functional/Magento/FunctionalTest/_generated/testManifest.txt`.<br/> You can split the list into multiple groups using `--config=parallel`; the groups will be generated in `_generated/groups/` like `_generated/groups/group1.txt, group2.txt, ...`.</br> Available values: `default` (default), `singleRun`(same as `default`), and `parallel`.</br> Example: `generate:tests --config=parallel`. |
122
+
|`--force`| Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
123
+
|`-i,--time`| Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
124
+
|`--tests`| Defines the test configuration as a JSON string.|
125
+
|`--debug`| Returns additional debug information (such as the filename where an error occurred) when test generation fails because of an invalid XML schema. This parameter takes extra processing time. Use it after test generation has failed once. |
126
+
|`-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `generate:tests SampleTest --remove` cleans up the entire `_generated` directory and generates `SampleTest` only.|
126
127
127
128
#### Examples of the JSON configuration
128
129
@@ -131,7 +132,7 @@ The configuration to generate a single test with no suites:
131
132
```json
132
133
{
133
134
"tests":[
134
-
"general_test1"//Generate the "general_test1" test.
135
+
"general_test1"//Generate the "general_test1" test.
135
136
],
136
137
"suites": null
137
138
}
@@ -141,10 +142,10 @@ The configuration to generate a single test in the suite:
141
142
142
143
```json
143
144
{
144
-
"tests": null, // No tests outside the suite configuration will be generated.
145
+
"tests": null, // No tests outside the suite configuration will be generated.
145
146
"suites":{
146
-
"sample":[ // The suite that contains the test.
147
-
"suite_test1"// The test to be generated.
147
+
"sample":[ // The suite that contains the test.
148
+
"suite_test1"// The test to be generated.
148
149
]
149
150
}
150
151
}
@@ -159,11 +160,11 @@ Complex configuration to generate a few non-suite tests, a single test in a suit
159
160
"general_test2",
160
161
"general_test3"
161
162
],
162
-
"suites":{ //Go to suites.
163
-
"sample":[ //Go to the "sample" suite.
164
-
"suite_test1"//Generate the "suite_test1" test.
163
+
"suites":{ //Go to suites.
164
+
"sample":[ //Go to the "sample" suite.
165
+
"suite_test1"//Generate the "suite_test1" test.
165
166
],
166
-
"sample2":[] //Generate all tests in the "sample2" suite.
167
+
"sample2":[] //Generate all tests in the "sample2" suite.
|`-r,--remove`| Removes the existing generated suites and tests cleaning up the `_generated` directory before the actual run. For example, `vendor/bin/mftf generate:suite WYSIWYG --remove` cleans up the entire `_generated` directory and generates `WYSIWYG` only. |
Copy file name to clipboardExpand all lines: docs/tips-tricks.md
+50Lines changed: 50 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -283,6 +283,27 @@ Use numbers within `stepKeys` when order is important, such as with testing sort
283
283
284
284
## Selectors
285
285
286
+
### Use contains() around text()
287
+
288
+
When possible, use `contains(text(), 'someTextHere')` rather than `text()='someTextHere'`.
289
+
`contains()` ignores whitespace while `text()` accounts for it.
290
+
291
+
**Why?**
292
+
If you are comparing text within a selector and have an unexpected space, or a blank line above or below the string, `text()` will fail while the `contains(text())` format will catch it.
293
+
In this scenario `text()` is more exacting. Use it when you need to be very precise about what is getting compared.
294
+
295
+
<spanstyle="color:green">
296
+
GOOD:
297
+
</span>
298
+
299
+
`//span[contains(text(), 'SomeTextHere')]`
300
+
301
+
<spanstyle="color:red">
302
+
BAD:
303
+
</span>
304
+
305
+
`//span[text()='SomeTextHere']`
306
+
286
307
### Build selectors in proper order
287
308
288
309
When building selectors for form elements, start with the parent context of the form element.
@@ -353,6 +374,31 @@ BAD:
353
374
354
375
## General tips
355
376
377
+
### Use data references to avoid hardcoded values
378
+
379
+
If you need to run a command such as `<magentoCLI command="config:set" />`, do not hardcode paths and values to the command.
380
+
Rather, create an appropriate `ConfigData.xml` file, which contains the required parameters for running the command.
0 commit comments