Skip to content

Mqe 2221: Merge master back into develop #759

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 39 commits into from
Jul 16, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
7de33e7
Small fix to the Mftf Getting Started doc
karyna-t Mar 2, 2020
5e44320
Merge pull request #611 from karyna-tsymbal-atwix/karyna-tsymbal-atwi…
okolesnyk Mar 4, 2020
7327be1
Update filter documentation with special characters to be able to sho…
okolesnyk Mar 4, 2020
ab4dfb4
DevDocs: Test merging precedence
lbajsarowicz Mar 11, 2020
84303c9
Adding the Selenium server download in the example
rafaelstz Mar 12, 2020
65fa3cc
Getting started > Removing code snippet
rafaelstz Mar 12, 2020
8df17bf
DevDocs: Update Best Practices
lbajsarowicz Mar 11, 2020
716ebd5
Add missing information about Backend Domain support.
lbajsarowicz Mar 10, 2020
984cc27
DevDocs: Update documentation regarding Versioning
lbajsarowicz Mar 10, 2020
84d2074
DevDocs: Update documentation regarding Update process
lbajsarowicz Mar 10, 2020
c161ab1
Update Introduction for MFTF
lbajsarowicz Mar 12, 2020
d236e2d
Merge branch 'pr625' into lbajsarowicz-doc-rollup
KevinBKozan Mar 13, 2020
63d7ab0
Merge branch 'pr626' into lbajsarowicz-doc-rollup
KevinBKozan Mar 13, 2020
4376c4b
Merge branch 'pr627' into lbajsarowicz-doc-rollup
KevinBKozan Mar 13, 2020
d654bcb
Merge branch 'pr628' into lbajsarowicz-doc-rollup
KevinBKozan Mar 13, 2020
03bbdb2
Merge branch 'pr629' into lbajsarowicz-doc-rollup
KevinBKozan Mar 13, 2020
11955b0
Editorial pass
dobooth Mar 13, 2020
87d1cee
Merge pull request #641 from magento/lbajsarowicz-doc-rollup
okolesnyk Mar 19, 2020
c09d463
Escaped Liquid tag syntax
dobooth Mar 19, 2020
5f67481
Merge pull request #646 from magento/dobooth-patch-1
KevinBKozan Mar 19, 2020
20cecfd
Fix an issue with MimeType guesser and latest release of "symfony/htt…
okolesnyk Mar 27, 2020
d254019
Fix an issue with MimeType guesser and latest release of "symfony/htt…
okolesnyk Mar 27, 2020
e45ea37
MQE-2052: MFTF uses undeclared dependency
okolesnyk Mar 27, 2020
2fe1372
Merge pull request #659 from magento/2.6.4-develop
okolesnyk Mar 27, 2020
b5d44a1
Merge pull request #638 from rafaelstz/patch-2
okolesnyk Apr 23, 2020
757e136
Merge pull request #639 from rafaelstz/patch-3
okolesnyk Apr 23, 2020
d839076
MQE-2210: Merge DevDoc changes in master back into MFTF release branc…
tomreece Jul 8, 2020
c663877
MQE-2203: Create CHANGELOG.MD entry for 3.0.0 (#748)
soumyau Jun 26, 2020
2149401
MQE-2206: CHANGELOG.md and composer version bumped to 3.0.0
jilu1 Jul 8, 2020
fd95a2a
MQE-2206: CHANGELOG.MD and Composer version bump
soumyau Jul 8, 2020
9a14b08
Merge pull request #752 from magento/MQE-2206
jilu1 Jul 8, 2020
62a19e7
Merge remote-tracking branch 'origin/3.0.0-GA' into MQE-2207
jilu1 Jul 9, 2020
26b0d6e
MQE-2207: merge 3.0.0-GA release branch to master
jilu1 Jul 9, 2020
a1508b6
Merge pull request #753 from magento/MQE-2207
jilu1 Jul 9, 2020
4cc0285
Remove links temporarily
tomreece Jul 9, 2020
3211363
Add back links
tomreece Jul 9, 2020
8d98efa
Merge pull request #754 from magento/quick-doc-fix
jilu1 Jul 9, 2020
9c8ab08
Set theme jekyll-theme-hacker
okolesnyk Jul 13, 2020
d8b1111
Merge remote-tracking branch 'origin/master' into sync-master-to-develop
jilu1 Jul 14, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,12 @@ Magento Functional Testing Framework Changelog
* [MFTF 3.0.0 RC2](https://www.youtube.com/watch?v=BJOQAw6dX5o)
* [MFTF 3.0.0 RC3](https://www.youtube.com/watch?v=scLb7pi8pR0)

2.6.4
-----

### Fixes
* added dependency to packages MFTF used but never specified in composer.json

2.6.3
-----

Expand Down
1 change: 1 addition & 0 deletions _config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
theme: jekyll-theme-hacker
2 changes: 1 addition & 1 deletion bin/mftf
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ try {
$version = $version['version'];
$application = new Symfony\Component\Console\Application();
$application->setName('Magento Functional Testing Framework CLI');
$application->setVersion('3.0.0');
$application->setVersion($version);
/** @var \Magento\FunctionalTestingFramework\Console\CommandListInterface $commandList */
$commandList = new \Magento\FunctionalTestingFramework\Console\CommandList;
foreach ($commandList->getCommands() as $command) {
Expand Down
3 changes: 2 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
"name": "magento/magento2-functional-testing-framework",
"description": "Magento2 Functional Testing Framework",
"type": "library",
"version": "3.0.0-RC5",
"version": "3.0.0",
"license": "AGPL-3.0",
"keywords": ["magento", "automation", "functional", "testing"],
"config": {
Expand Down Expand Up @@ -30,6 +30,7 @@
"spomky-labs/otphp": "^10.0",
"symfony/console": "^4.4",
"symfony/finder": "^5.0",
"symfony/http-foundation": "^5.0",
"symfony/mime": "^5.0",
"symfony/process": "^4.4",
"vlucas/phpdotenv": "^2.4",
Expand Down
147 changes: 134 additions & 13 deletions composer.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 4 additions & 4 deletions docs/backward-incompatible-changes.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ We removed support to read test modules from the deprecated path `dev/tests/acce
- Files under test modules `ActionGroup`, `Page`, `Section`, `Test` and `Suite` only support a single entity per file.
- The `file` attribute from `<module>` has been removed from the suite schema. `<module file=""/>` is no longer supported in suites.
- Metadata filename format changed to ***`*Meta.xml`***.
- Only nested assertion syntax will be supported. See the [assertions page](./docs/test/assertions.md) for details. Here is an example of the nested assertion syntax:
- Only nested assertion syntax will be supported. See the [assertions page](test/assertions.md) for details. Here is an example of the nested assertion syntax:
```xml
<assertEquals stepKey="assertAddressOrderPage">
<actualResult type="const">$billingAddressOrderPage</actualResult>
Expand All @@ -40,7 +40,7 @@ To run the upgrade tests:

1. Run `bin/mftf reset --hard` to remove old generated configurations.
1. Run `bin/mftf build:project` to generate new configurations.
1. Run `bin/mftf upgrade:tests`. [See command page for details](./docs/commands/mftf.md#upgradetests).
1. Run `bin/mftf upgrade:tests`. [See command page for details](commands/mftf.md#upgradetests).
1. Lastly, try to generate all tests. Tests should all be generated as a result of the upgrades. If not, the most likely issue will be a changed XML schema. Check error messaging and search your codebase for the attributes listed.

## MFTF commands
Expand All @@ -57,7 +57,7 @@ To run the upgrade tests:

**Details**:

The `helper` allows test writers to solve advanced requirements beyond what MFTF offers out of the box. See [custom-helpers](./docs/custom-helpers.md) for more information on usage.
The `helper` allows test writers to solve advanced requirements beyond what MFTF offers out of the box. See [custom-helpers](custom-helpers.md) for more information on usage.

Here is an example of using `helper` in place of `executeSelenium` to achieve same workflow.

Expand Down Expand Up @@ -95,7 +95,7 @@ New usage:

**Details**:

See the [actions page for details](./docs/test/actions.md#pause). Here is a usage example:
See the [actions page for details](test/actions.md#pause). Here is a usage example:

```xml
<pause stepKey="pauseExecutionKey"/>
Expand Down
66 changes: 53 additions & 13 deletions docs/best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,49 @@

Check out our best practices below to ensure you are getting the absolute most out of the Magento Functional Testing Framework.

## Focus on reusability

### Use existing Tests and resources

Magento offers more than **3000** acceptance tests, **2500** [Action group]s, **750** Page declarations with more than **1500** Section definitions.
It is very probable that behaviour you want to test already exists as a Test or Action Group.
Instead of writing everything by yourself - use `extends` attribute to refer to existing element and customize it.

**Reusable Resources**

{%raw%}

* Tests (reusable with `<test extends="...">` argument)
* Action Group (reusable with including `<actionGroup ref="...">`, or extending `<actionGroup extends="...">`)
* Pages (reusable with reference `{{PageDefinition.url}}`)
* Sections (reusable with reference `{{SectionDefinition.elementDefinition}}`)
* Data Entities (reusable with reference `<createData entity="...">"` or extending `<entity extends="...">`)

{%endraw%}

<div class="bs-callout bs-callout-warning" markdown="1">

Avoid using resources that are marked as **Deprecated**. Usually there is a replacement provided for a deprecated resource.

</div>

### Extract repetitive Actions

Instead of writing a few of Tests that perform mostly the same actions, you should thing about [Action group] that is a container for repetitive Actions.
If each run needs different data, use `<arguments>` to inject necessary information.

We recommend to keep Action Groups having single responsibility, for example `AdminLoginActionGroup`, which expected outcome is being logged in as Administrator when [Action group] is executed.

## Contribute

Althought the Magento Core team and Contributors join forces to cover most of the features with tests, it is impossible to have this done quickly.
If you've covered Magento Core feature with Functional Tests - you are more than welcome to contribute.

You can also help with MFTF Test Migration to get the experience and valuable feedback from other community members and maintainers.

## Action group

1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used.
Add additional explanation in annotations if needed.
1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used. Add additional explanation in annotations if needed.
2. Provide default values for the arguments that apply to your most common case scenarios.
3. One `<actionGroup>` tag is allowed per action group XML file.

Expand Down Expand Up @@ -95,20 +134,20 @@ Use the _Foo.camelCase_ naming convention, which is similar to _Classes_ and _cl

Use an upper case first letter for:

- File names. Example: _StorefrontCreateCustomerTest.xml_
- Test name attributes. Example: `<test name="TestAllTheThingsTest">`
- Data entity names. Example: `<entity name="OutOfStockProduct">`
- Page name. Example: `<page name="AdminLoginPage">`
- Section name. Example: `<section name="AdminCategorySidebarActionSection">`
- Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`
* File names. Example: _StorefrontCreateCustomerTest.xml_
* Test name attributes. Example: `<test name="TestAllTheThingsTest">`
* Data entity names. Example: `<entity name="OutOfStockProduct">`
* Page name. Example: `<page name="AdminLoginPage">`
* Section name. Example: `<section name="AdminCategorySidebarActionSection">`
* Action group name. Example: `<actionGroup name="LoginToAdminActionGroup">`

#### Lower case

Use a lower case first letter for:

- Data keys. Example: `<data key="firstName">`
- Element names. Examples: `<element name="confirmDeleteButton"/>`
- Step keys. For example: `<click selector="..." stepKey="clickLogin"/>`
* Data keys. Example: `<data key="firstName">`
* Element names. Examples: `<element name="confirmDeleteButton"/>`
* Step keys. For example: `<click selector="..." stepKey="clickLogin"/>`

## Page object

Expand Down Expand Up @@ -147,9 +186,9 @@ Define these three elements and reference them by name in the tests.
1. Keep your tests short and granular for target testing, easier reviews, and easier merge conflict resolution.
It also helps you to identify the cause of test failure.
1. Use comments to keep tests readable and maintainable:
- Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
* Keep the inline `<!-- XML comments -->` and [`<comment>`] tags up to date.
It helps to inform the reader of what you are testing and to yield a more descriptive Allure report.
- Explain in comments unclear or tricky test steps.
* Explain in comments unclear or tricky test steps.
1. Refer to [sections] instead of writing selectors.
1. One `<test>` tag is allowed per test XML file.

Expand Down Expand Up @@ -178,3 +217,4 @@ Since the configurable product module could be disabled, this approach is more r
[merging]: merging.html
[parameterized selectors]: section/parameterized-selectors.html
[sections]: section.html
[MFTF Test Migration]: https://github.com/magento/magento-functional-tests-migration
2 changes: 1 addition & 1 deletion docs/commands/mftf.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ vendor/bin/mftf generate:tests [option] [<test name>] [<test name>] [--remove]
| Option | Description|
| ---| --- |
| `--config=[<default> or <singleRun> or <parallel>]` | Creates a single manifest file with a list of all tests. The default location is `tests/functional/Magento/FunctionalTest/_generated/testManifest.txt`.<br/> You can split the list into multiple groups using `--config=parallel`; the groups will be generated in `_generated/groups/` like `_generated/groups/group1.txt, group2.txt, ...`.<br/> Available values: `default` (default), `singleRun`(same as `default`), and `parallel`.<br/> Example: `generate:tests --config=parallel`. |
| `--filter` | Option to filter tests to be generated.<br/>Template: '<filterName>:<filterValue>'.<br/>Existing filter types: severity.<br/>Existing severity values: BLOCKER, CRITICAL, MAJOR, AVERAGE, MINOR.<br/>Example: --filter=severity:CRITICAL|
| `--filter` | Option to filter tests to be generated.<br/>Template: '&lt;filterName&gt;:&lt;filterValue&gt;'.<br/>Existing filter types: severity.<br/>Existing severity values: BLOCKER, CRITICAL, MAJOR, AVERAGE, MINOR.<br/>Example: --filter=severity:CRITICAL|
| `--force` | Forces test generation, regardless of the module merge order defined in the Magento instance. Example: `generate:tests --force`. |
| `-i,--time` | Set time in minutes to determine the group size when `--config=parallel` is used. The __default value__ is `10`. Example: `generate:tests --config=parallel --time=15`|
| `--tests` | Defines the test configuration as a JSON string.|
Expand Down
Loading