From f11b3f5c4d58b99cdc90c75763b9e94613d853cb Mon Sep 17 00:00:00 2001 From: Soumya Unnikrishnan Date: Tue, 26 May 2020 17:08:15 -0500 Subject: [PATCH 1/4] MQE-2141: MFTF BIC Documentation --- docs/backward-incompatible-changes.md | 151 ++++++++++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 docs/backward-incompatible-changes.md diff --git a/docs/backward-incompatible-changes.md b/docs/backward-incompatible-changes.md new file mode 100644 index 000000000..500f4883a --- /dev/null +++ b/docs/backward-incompatible-changes.md @@ -0,0 +1,151 @@ +# MFTF 3.0.0 backward incompatible changes + +This page highlights backward incompatible changes between releases that have a major impact and require detailed explanation and special instructions to ensure third-party tests continue working with Magento core tests. + +## Version requirement changes + +We changed the minimum PHP version requirement from 7.0 to 7.3. Because of the PHP version requirement change, this MFTF version supports only Magento 2.4 or later. + +## Folder structure changes + +We removed support to read test modules from deprecated path `dev/tests/acceptance/tests/functional/Magento/FunctionalTest`. If there are test modules in this path, they would need to be moved to `dev/tests/acceptance/tests/functional/Magento`. + +## XSD schema changes + +- Files under test modules `ActionGroup`, `Page`, `Section`, `Test` and `Suite` support only a single entity per file. + +- `file` attribute from `` has been removed from suite schema. `` is no longer supported in suites. + +- Metadata filename format changed to ***`*Meta.xml`***. + +- Only nested assertion syntax will be supported. [See assertions page for details](./docs/test/assertions.md). Here is an example of a nested assertion syntax. +```xml + + $billingAddressOrderPage + $shippingAddressOrderPage + +``` +### Upgrading tests to new schema + +The following table lists the upgrade scripts that are available to upgrade tests to the new schema. + +| Script name | Description | +|-----------------------|-----------------------------------------------------------------------------------------------------------| +|`splitMultipleEntitiesFiles`| Splits files that have multiple entities into multiple files with one entity per file. | +|`upgradeAssertionSchema`| Updates assert actions that use old assertion syntax to new nested syntax.| +|`renameMetadataFiles`| Renames Metadata filenames to `*Meta.xml`.| +|`removeModuleFileInSuiteFiles`| Removes occurrences of `` from all ``s.| +|`removeUnusedArguments`| Removes unused arguments from action groups.| +|`upgradeTestSchema`| Replaces relative schema paths to URN in test files.| + +Here's how you can upgrade tests: + +- Run `bin/mftf reset --hard` to remove old generated configurations. +- Run `bin/mftf build:project` to generate new configurations. +- Run `bin/mftf upgrade:tests`. [See command page for details](./docs/commands/mftf.md#upgradetests). +- Lastly, try to generate all tests. Tests should all be generated as a result of the upgrades. If not, the most likely issue will be a changed XML schema. Check error messaging and search your codebase for the attributes listed. + +## MFTF commands + +`--debug` option `NONE` removed for strict schema validation. Ensure there are no schema validation errors in test modules before running MFTF commands. + +## MFTF actions + +###`executeInSelenium` and `performOn` removed + +**Action**: Deprecated actions `executeInSelenium` and `performOn` are removed in favor of new action `helper`. + +**Reason**: `executeInSelenium` and `performOn` allowed custom PHP code to be written inline inside of XML files which was difficult to maintain, troubleshoot, and modify. + +**Details**: `helper` will allow test writers to solve advanced requirements beyond what MFTF offers out of the box.[See custom-helpers](./docs/custom-helpers.md) for more information on the usage. + +Here's an example of using `helper` instead of `executeSelenium` to achieve same workflow. + +Old usage: +```xml + +``` + +New usage: +```xml + + //div[contains(@class, 'inline-wysiwyg')]//h2 + {{TinyMCEPartialHeadingSelection.startX}} + {{TinyMCEPartialHeadingSelection.startY}} + {{TinyMCEPartialHeadingSelection.endX}} + {{TinyMCEPartialHeadingSelection.endY}} + +``` +### `pauseExecution` removed + +**Action**: `pauseExecution` is removed in favor of `pause`. + +**Reason**: `[WebDriver]pauseExecution` is removed in Codeception 3 in favor of `I->pause()`. + +**Description**: [See actions page for details](./docs/test/actions.md#pause). Here's a usage example. +```xml + +``` + +### Removed assert actions + +**Action**: Assert actions `assertInternalType`, `assertNotInternalType` and `assertArraySubset` are removed. + +**Reason**: PHPUnit 9 has dropped support for these assertions. + +### Updated assert actions + +**Action**: `delta` attribute has been removed from `assertEquals` and `assertNotEquals`. Instead, below assert actions have been introduced: + - `assertEqualsWithDelta` + - `assertNotEqualsWithDelta` + - `assertEqualsCanonicalizing` + - `assertNotEqualsCanonicalizing` + - `assertEqualsIgnoringCase` + - `assertNotEqualsIgnoringCase` + +**Reason**: PHPUnit 9 has dropped support for optional parameters for `assertEquals` and `assertNotEquals` and has introduced these new assertions. + +**Description**: Usages of `assertEquals` or `assertNotEquals` with `delta` specified, should be replaced with appropriate assertion from above list. + +### `assertContains` supports only iterable haystacks + +**Action**: `assertContains` and `assertNotContains` now support only iterable haystacks. Below assert actions have been added to work with string haystacks: +- `assertStringContainsString` +- `assertStringNotContainsString` +- `assertStringContainsStringIgnoringCase` +- `assertStringNotContainsStringIgnoringCase` + +**Reason**: With PHPUnit 9, `assertContains` and `assertNotContains` only allows iterable haystacks. New assertions have been introduced to support string haystacks. + +**Description**: Usages of `assertContains` and `assertNotContains` with string haystacks should be replaced with appropriate assertion from above list. + +Usage example for string haystacks: +```xml + +$grabSimpleProdPrice2 +$110.70 + +``` + +### `formatMoney` removed + +**Action**: `formatMoney` has been removed in favor of `formatCurrency`. + +**Reason**: PHP 7.4 has deprecated use of `formatMoney`. + +**Description**: Format input to specified currency according to the locale specified. + +Usage example: +```xml + +``` + + From a411c0cbb881f979b7b4c0ac9f7810e976deae5e Mon Sep 17 00:00:00 2001 From: Soumya Unnikrishnan Date: Tue, 26 May 2020 17:18:14 -0500 Subject: [PATCH 2/4] MQE-2141: MFTF BIC Documentation --- docs/backward-incompatible-changes.md | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/docs/backward-incompatible-changes.md b/docs/backward-incompatible-changes.md index 500f4883a..fe529619a 100644 --- a/docs/backward-incompatible-changes.md +++ b/docs/backward-incompatible-changes.md @@ -57,9 +57,11 @@ Here's how you can upgrade tests: **Reason**: `executeInSelenium` and `performOn` allowed custom PHP code to be written inline inside of XML files which was difficult to maintain, troubleshoot, and modify. -**Details**: `helper` will allow test writers to solve advanced requirements beyond what MFTF offers out of the box.[See custom-helpers](./docs/custom-helpers.md) for more information on the usage. +**Details**: -Here's an example of using `helper` instead of `executeSelenium` to achieve same workflow. +`helper` will allow test writers to solve advanced requirements beyond what MFTF offers out of the box.[See custom-helpers](./docs/custom-helpers.md) for more information on the usage. + +Here's an example of using `helper` in place of `executeSelenium` to achieve same workflow. Old usage: ```xml @@ -84,13 +86,16 @@ New usage: {{TinyMCEPartialHeadingSelection.endY}} ``` + ### `pauseExecution` removed **Action**: `pauseExecution` is removed in favor of `pause`. **Reason**: `[WebDriver]pauseExecution` is removed in Codeception 3 in favor of `I->pause()`. -**Description**: [See actions page for details](./docs/test/actions.md#pause). Here's a usage example. +**Details**: + +[See actions page for details](./docs/test/actions.md#pause). Here's a usage example. ```xml ``` @@ -113,7 +118,9 @@ New usage: **Reason**: PHPUnit 9 has dropped support for optional parameters for `assertEquals` and `assertNotEquals` and has introduced these new assertions. -**Description**: Usages of `assertEquals` or `assertNotEquals` with `delta` specified, should be replaced with appropriate assertion from above list. +**Details**: + +Usages of `assertEquals` or `assertNotEquals` with `delta` specified, should be replaced with appropriate assertion from above list. ### `assertContains` supports only iterable haystacks @@ -125,7 +132,9 @@ New usage: **Reason**: With PHPUnit 9, `assertContains` and `assertNotContains` only allows iterable haystacks. New assertions have been introduced to support string haystacks. -**Description**: Usages of `assertContains` and `assertNotContains` with string haystacks should be replaced with appropriate assertion from above list. +**Details**: + +Usages of `assertContains` and `assertNotContains` with string haystacks should be replaced with appropriate assertion from above list. Usage example for string haystacks: ```xml @@ -141,11 +150,9 @@ Usage example for string haystacks: **Reason**: PHP 7.4 has deprecated use of `formatMoney`. -**Description**: Format input to specified currency according to the locale specified. +**Details**: Format input to specified currency according to the locale specified. Usage example: ```xml ``` - - From aea82ab1b307b0ad25c49467c713800927acf58f Mon Sep 17 00:00:00 2001 From: Donald Booth Date: Wed, 27 May 2020 10:04:47 -0500 Subject: [PATCH 3/4] Grammar and formatting --- docs/backward-incompatible-changes.md | 87 ++++++++++++++------------- 1 file changed, 46 insertions(+), 41 deletions(-) diff --git a/docs/backward-incompatible-changes.md b/docs/backward-incompatible-changes.md index fe529619a..c50f490ca 100644 --- a/docs/backward-incompatible-changes.md +++ b/docs/backward-incompatible-changes.md @@ -4,46 +4,44 @@ This page highlights backward incompatible changes between releases that have a ## Version requirement changes -We changed the minimum PHP version requirement from 7.0 to 7.3. Because of the PHP version requirement change, this MFTF version supports only Magento 2.4 or later. +We changed the minimum PHP version requirement from 7.0 to 7.3. Because of the PHP version requirement change, this MFTF version only supports Magento 2.4 or later. ## Folder structure changes -We removed support to read test modules from deprecated path `dev/tests/acceptance/tests/functional/Magento/FunctionalTest`. If there are test modules in this path, they would need to be moved to `dev/tests/acceptance/tests/functional/Magento`. +We removed support to read test modules from the deprecated path `dev/tests/acceptance/tests/functional/Magento/FunctionalTest`. If there are test modules in this path, they should be moved to `dev/tests/acceptance/tests/functional/Magento`. ## XSD schema changes -- Files under test modules `ActionGroup`, `Page`, `Section`, `Test` and `Suite` support only a single entity per file. +- Files under test modules `ActionGroup`, `Page`, `Section`, `Test` and `Suite` only support a single entity per file. +- The `file` attribute from `` has been removed from the suite schema. `` is no longer supported in suites. +- Metadata filename format changed to ***`*Meta.xml`***. +- Only nested assertion syntax will be supported. See the [assertions page](./docs/test/assertions.md) for details. Here is an example of the nested assertion syntax: + ```xml + + $billingAddressOrderPage + $shippingAddressOrderPage + + ``` -- `file` attribute from `` has been removed from suite schema. `` is no longer supported in suites. - -- Metadata filename format changed to ***`*Meta.xml`***. - -- Only nested assertion syntax will be supported. [See assertions page for details](./docs/test/assertions.md). Here is an example of a nested assertion syntax. -```xml - - $billingAddressOrderPage - $shippingAddressOrderPage - -``` -### Upgrading tests to new schema +### Upgrading tests to the new schema The following table lists the upgrade scripts that are available to upgrade tests to the new schema. | Script name | Description | |-----------------------|-----------------------------------------------------------------------------------------------------------| |`splitMultipleEntitiesFiles`| Splits files that have multiple entities into multiple files with one entity per file. | -|`upgradeAssertionSchema`| Updates assert actions that use old assertion syntax to new nested syntax.| +|`upgradeAssertionSchema`| Updates assert actions that uses the old assertion syntax into the new nested syntax.| |`renameMetadataFiles`| Renames Metadata filenames to `*Meta.xml`.| |`removeModuleFileInSuiteFiles`| Removes occurrences of `` from all ``s.| |`removeUnusedArguments`| Removes unused arguments from action groups.| |`upgradeTestSchema`| Replaces relative schema paths to URN in test files.| -Here's how you can upgrade tests: +To run the upgrade tests: -- Run `bin/mftf reset --hard` to remove old generated configurations. -- Run `bin/mftf build:project` to generate new configurations. -- Run `bin/mftf upgrade:tests`. [See command page for details](./docs/commands/mftf.md#upgradetests). -- Lastly, try to generate all tests. Tests should all be generated as a result of the upgrades. If not, the most likely issue will be a changed XML schema. Check error messaging and search your codebase for the attributes listed. +1. Run `bin/mftf reset --hard` to remove old generated configurations. +1. Run `bin/mftf build:project` to generate new configurations. +1. Run `bin/mftf upgrade:tests`. [See command page for details](./docs/commands/mftf.md#upgradetests). +1. Lastly, try to generate all tests. Tests should all be generated as a result of the upgrades. If not, the most likely issue will be a changed XML schema. Check error messaging and search your codebase for the attributes listed. ## MFTF commands @@ -51,7 +49,7 @@ Here's how you can upgrade tests: ## MFTF actions -###`executeInSelenium` and `performOn` removed +### `executeInSelenium` and `performOn` removed **Action**: Deprecated actions `executeInSelenium` and `performOn` are removed in favor of new action `helper`. @@ -59,11 +57,12 @@ Here's how you can upgrade tests: **Details**: -`helper` will allow test writers to solve advanced requirements beyond what MFTF offers out of the box.[See custom-helpers](./docs/custom-helpers.md) for more information on the usage. +The `helper` allows test writers to solve advanced requirements beyond what MFTF offers out of the box. See [custom-helpers](./docs/custom-helpers.md) for more information on usage. -Here's an example of using `helper` in place of `executeSelenium` to achieve same workflow. +Here is an example of using `helper` in place of `executeSelenium` to achieve same workflow. Old usage: + ```xml //div[contains(@class, 'inline-wysiwyg')]//h2 @@ -95,7 +95,8 @@ New usage: **Details**: -[See actions page for details](./docs/test/actions.md#pause). Here's a usage example. +See the [actions page for details](./docs/test/actions.md#pause). Here is a usage example: + ```xml ``` @@ -108,39 +109,42 @@ New usage: ### Updated assert actions -**Action**: `delta` attribute has been removed from `assertEquals` and `assertNotEquals`. Instead, below assert actions have been introduced: - - `assertEqualsWithDelta` - - `assertNotEqualsWithDelta` - - `assertEqualsCanonicalizing` - - `assertNotEqualsCanonicalizing` - - `assertEqualsIgnoringCase` - - `assertNotEqualsIgnoringCase` +**Action**: The `delta` attribute has been removed from `assertEquals` and `assertNotEquals`. Instead, new assert actions have been introduced: + + - `assertEqualsWithDelta` + - `assertNotEqualsWithDelta` + - `assertEqualsCanonicalizing` + - `assertNotEqualsCanonicalizing` + - `assertEqualsIgnoringCase` + - `assertNotEqualsIgnoringCase` **Reason**: PHPUnit 9 has dropped support for optional parameters for `assertEquals` and `assertNotEquals` and has introduced these new assertions. **Details**: -Usages of `assertEquals` or `assertNotEquals` with `delta` specified, should be replaced with appropriate assertion from above list. +Usage of `assertEquals` or `assertNotEquals` with a specified `delta`, should be replaced with appropriate assertion from the above list. ### `assertContains` supports only iterable haystacks -**Action**: `assertContains` and `assertNotContains` now support only iterable haystacks. Below assert actions have been added to work with string haystacks: -- `assertStringContainsString` -- `assertStringNotContainsString` -- `assertStringContainsStringIgnoringCase` -- `assertStringNotContainsStringIgnoringCase` +**Action**: `assertContains` and `assertNotContains` now only supports iterable haystacks. These assert actions have been added to work with string haystacks: + +- `assertStringContainsString` +- `assertStringNotContainsString` +- `assertStringContainsStringIgnoringCase` +- `assertStringNotContainsStringIgnoringCase` **Reason**: With PHPUnit 9, `assertContains` and `assertNotContains` only allows iterable haystacks. New assertions have been introduced to support string haystacks. **Details**: -Usages of `assertContains` and `assertNotContains` with string haystacks should be replaced with appropriate assertion from above list. +Usages of `assertContains` and `assertNotContains` with string haystacks should be replaced with appropriate assertion from the above list. Usage example for string haystacks: + ```xml -$grabSimpleProdPrice2 -$110.70 + $grabSimpleProdPrice2 + $110.70 ``` @@ -153,6 +157,7 @@ Usage example for string haystacks: **Details**: Format input to specified currency according to the locale specified. Usage example: + ```xml ``` From 482d2b0886c619ffd480cbe48c2e9aa527518fd2 Mon Sep 17 00:00:00 2001 From: Soumya Unnikrishnan Date: Mon, 1 Jun 2020 10:05:58 -0500 Subject: [PATCH 4/4] MQE-2141: MFTF BIC Documentation addressed review comments --- docs/backward-incompatible-changes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/backward-incompatible-changes.md b/docs/backward-incompatible-changes.md index c50f490ca..746e4546b 100644 --- a/docs/backward-incompatible-changes.md +++ b/docs/backward-incompatible-changes.md @@ -2,7 +2,7 @@ This page highlights backward incompatible changes between releases that have a major impact and require detailed explanation and special instructions to ensure third-party tests continue working with Magento core tests. -## Version requirement changes +## Minimum supported PHP version changes We changed the minimum PHP version requirement from 7.0 to 7.3. Because of the PHP version requirement change, this MFTF version only supports Magento 2.4 or later.