Merge pull request #759 from magento/MQE-2221

Mqe 2221: Merge master back into develop

Author: jilu1

CHANGELOG.md

@@ -88,6 +88,12 @@ Magento Functional Testing Framework Changelog
 * [MFTF 3.0.0 RC2](https://www.youtube.com/watch?v=BJOQAw6dX5o)
 * [MFTF 3.0.0 RC3](https://www.youtube.com/watch?v=scLb7pi8pR0)
 
+2.6.4
+-----
+
+### Fixes
+* added dependency to packages MFTF used but never specified in composer.json
+
 2.6.3
 -----
 

_config.yml

@@ -0,0 +1 @@
+theme: jekyll-theme-hacker

bin/mftf

@@ -31,7 +31,7 @@ try {
     $version = $version['version'];
     $application = new Symfony\Component\Console\Application();
     $application->setName('Magento Functional Testing Framework CLI');
-    $application->setVersion('3.0.0');
+    $application->setVersion($version);
     /** @var \Magento\FunctionalTestingFramework\Console\CommandListInterface $commandList */
     $commandList = new \Magento\FunctionalTestingFramework\Console\CommandList;
     foreach ($commandList->getCommands() as $command) {

composer.json

@@ -2,7 +2,7 @@
     "name": "magento/magento2-functional-testing-framework",
     "description": "Magento2 Functional Testing Framework",
     "type": "library",
-    "version": "3.0.0-RC5",
+    "version": "3.0.0",
     "license": "AGPL-3.0",
     "keywords": ["magento", "automation", "functional", "testing"],
     "config": {
@@ -30,6 +30,7 @@
         "spomky-labs/otphp": "^10.0",
         "symfony/console": "^4.4",
         "symfony/finder": "^5.0",
+        "symfony/http-foundation": "^5.0",
         "symfony/mime": "^5.0",
         "symfony/process": "^4.4",
         "vlucas/phpdotenv": "^2.4",

composer.lock

@@ -15,7 +15,7 @@ We removed support to read test modules from the deprecated path `dev/tests/acce
 -  Files under test modules `ActionGroup`, `Page`, `Section`, `Test` and `Suite` only support a single entity per file. 
 -  The `file` attribute from `<module>` has been removed from the suite schema. `<module file=""/>` is no longer supported in suites.
 -  Metadata filename format changed to ***`*Meta.xml`***.
--  Only nested assertion syntax will be supported. See the [assertions page](./docs/test/assertions.md) for details. Here is an example of the nested assertion syntax:
+-  Only nested assertion syntax will be supported. See the [assertions page](test/assertions.md) for details. Here is an example of the nested assertion syntax:
    ```xml
    <assertEquals stepKey="assertAddressOrderPage">
       <actualResult type="const">$billingAddressOrderPage</actualResult>
@@ -40,7 +40,7 @@ To run the upgrade tests:
 
 1. Run `bin/mftf reset --hard` to remove old generated configurations.
 1. Run `bin/mftf build:project` to generate new configurations.
-1. Run `bin/mftf upgrade:tests`. [See command page for details](./docs/commands/mftf.md#upgradetests).
+1. Run `bin/mftf upgrade:tests`. [See command page for details](commands/mftf.md#upgradetests).
 1. Lastly, try to generate all tests. Tests should all be generated as a result of the upgrades. If not, the most likely issue will be a changed XML schema. Check error messaging and search your codebase for the attributes listed.
 
 ## MFTF commands
@@ -57,7 +57,7 @@ To run the upgrade tests:
 
 **Details**: 
 
-The `helper` allows test writers to solve advanced requirements beyond what MFTF offers out of the box. See [custom-helpers](./docs/custom-helpers.md) for more information on usage. 
+The `helper` allows test writers to solve advanced requirements beyond what MFTF offers out of the box. See [custom-helpers](custom-helpers.md) for more information on usage. 
 
 Here is an example of using `helper` in place of `executeSelenium` to achieve same workflow.
 
@@ -95,7 +95,7 @@ New usage:
 
 **Details**: 
 
-See the [actions page for details](./docs/test/actions.md#pause). Here is a usage example:
+See the [actions page for details](test/actions.md#pause). Here is a usage example:
 
 ```xml
 <pause stepKey="pauseExecutionKey"/>

docs/backward-incompatible-changes.md

@@ -2,10 +2,49 @@
 
 Check out our best practices below to ensure you are getting the absolute most out of the Magento Functional Testing Framework.
 
+## Focus on reusability
+
+### Use existing Tests and resources
+
+Magento offers more than **3000** acceptance tests, **2500** [Action group]s, **750** Page declarations with more than **1500** Section definitions.
+It is very probable that behaviour you want to test already exists as a Test or Action Group.
+Instead of writing everything by yourself - use `extends` attribute to refer to existing element and customize it.
+
+**Reusable Resources**
+
+{%raw%}
+
+*  Tests (reusable with `<test extends="...">` argument)
+*  Action Group (reusable with including `<actionGroup ref="...">`, or extending `<actionGroup extends="...">`)
+*  Pages (reusable with reference `{{PageDefinition.url}}`)
+*  Sections (reusable with reference `{{SectionDefinition.elementDefinition}}`)
+*  Data Entities (reusable with reference `<createData entity="...">"` or extending `<entity extends="...">`)
+
+{%endraw%}
+
+<div class="bs-callout bs-callout-warning" markdown="1">
+
+Avoid using resources that are marked as **Deprecated**. Usually there is a replacement provided for a deprecated resource.
+
+</div>
+
+### Extract repetitive Actions
+
+Instead of writing a few of Tests that perform mostly the same actions, you should thing about [Action group] that is a container for repetitive Actions.
+If each run needs different data, use `<arguments>` to inject necessary information.
+
+We recommend to keep Action Groups having single responsibility, for example `AdminLoginActionGroup`, which expected outcome is being logged in as Administrator when [Action group] is executed.
+
+## Contribute
+
+Althought the Magento Core team and Contributors join forces to cover most of the features with tests, it is impossible to have this done quickly.
+If you've covered Magento Core feature with Functional Tests - you are more than welcome to contribute.
+
+You can also help with MFTF Test Migration to get the experience and valuable feedback from other community members and maintainers.
+
 ## Action group
 
-1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used.
- Add additional explanation in annotations if needed.
+1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used. Add additional explanation in annotations if needed.
 2. Provide default values for the arguments that apply to your most common case scenarios.
 3. One `<actionGroup>` tag is allowed per action group XML file.
 
@@ -95,20 +134,20 @@ Use the _Foo.camelCase_ naming convention, which is similar to _Classes_ and _cl
 
 Use an upper case first letter for:
 
--  File names. Example: _StorefrontCreateCustomerTest.xml_
--  Test name attributes. Example: `<test name="TestAllTheThingsTest">`
--  Data entity names. Example: `<entity name="OutOfStockProduct">`
--  Page name. Example: `<page name="AdminLoginPage">`
--  Section name. Example: `<section name="AdminCategorySidebarActionSection">`
--  Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`
+*  File names. Example: _StorefrontCreateCustomerTest.xml_
+*  Test name attributes. Example: `<test name="TestAllTheThingsTest">`
+*  Data entity names. Example: `<entity name="OutOfStockProduct">`
+*  Page name. Example: `<page name="AdminLoginPage">`
+*  Section name. Example: `<section name="AdminCategorySidebarActionSection">`
+*  Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`
 
 #### Lower case
 
 Use a lower case first letter for:
 
--  Data keys. Example: `<data key="firstName">`
--  Element names. Examples: `<element name="confirmDeleteButton"/>`
--  Step keys. For example: `<click selector="..." stepKey="clickLogin"/>`
+*  Data keys. Example: `<data key="firstName">`
+*  Element names. Examples: `<element name="confirmDeleteButton"/>`
+*  Step keys. For example: `<click selector="..." stepKey="clickLogin"/>`
 
 ## Page object
 
@@ -147,9 +186,9 @@ Define these three elements and reference them by name in the tests.
 1. Keep your tests short and granular for target testing, easier reviews, and easier merge conflict resolution.
  It also helps you to identify the cause of test failure.
 1. Use comments to keep tests readable and maintainable:
-   -  Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
+   *  Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
      It helps to inform the reader of what you are testing and to yield a more descriptive Allure report.
-   -  Explain in comments unclear or tricky test steps.
+   *  Explain in comments unclear or tricky test steps.
 1. Refer to [sections] instead of writing selectors.
 1. One `<test>` tag is allowed per test XML file.
 
@@ -178,3 +217,4 @@ Since the configurable product module could be disabled, this approach is more r
 [merging]: merging.html
 [parameterized selectors]: section/parameterized-selectors.html
 [sections]: section.html
+[MFTF Test Migration]: https://github.com/magento/magento-functional-tests-migration

docs/best-practices.md

@@ -151,7 +151,7 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
 | Option | Description|
 | ---| --- |
 | `--config=[<default> or <singleRun> or <parallel>]` | Creates a single manifest file with a list of all tests. The default location is `tests/functional/Magento/FunctionalTest/_generated/testManifest.txt`.<br/> You can split the list into multiple groups using `--config=parallel`; the groups will be generated in `_generated/groups/` like `_generated/groups/group1.txt, group2.txt, ...`.<br/> Available values: `default` (default), `singleRun`(same as `default`), and `parallel`.<br/> Example: `generate:tests --config=parallel`. |
-| `--filter` | Option to filter tests to be generated.<br/>Template: '<filterName>:<filterValue>'.<br/>Existing filter types: severity.<br/>Existing severity values: BLOCKER, CRITICAL, MAJOR, AVERAGE, MINOR.<br/>Example: --filter=severity:CRITICAL|
+| `--filter` | Option to filter tests to be generated.<br/>Template: '&lt;filterName&gt;:&lt;filterValue&gt;'.<br/>Existing filter types: severity.<br/>Existing severity values: BLOCKER, CRITICAL, MAJOR, AVERAGE, MINOR.<br/>Example: --filter=severity:CRITICAL|
 | `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
 | `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
 | `--tests` | Defines the test configuration as a JSON string.|