Merge branch 'develop' into CLI-direct-shell-exec

Author: soumyau

.github/CONTRIBUTING.md

@@ -40,7 +40,7 @@ All contributors are required to submit a click-through form to agree to the ter
 
    - Unit/integration test coverage
    - Proposed documentation update.
-     For the documentation contribution guidelines, see [DevDocs Contributing][devdocs].
+     For the documentation contribution guidelines, see [DOCUMENTATION_TEMPLATE][].
 7. For large features or changes, [open an issue][issue] to discuss first.
    This may prevent duplicate or unnecessary effort, and it may gain you some additional contributors.
 8. To report a bug, [open an issue][issue], and follow [guidelines about bugfix issues][issue reporting].
@@ -160,7 +160,7 @@ Label| Description
 [create issue]: https://help.github.com/articles/creating-an-issue/
 [create pr]: https://help.github.com/articles/creating-a-pull-request/
 [Definition of Done]: https://devdocs.magento.com/guides/v2.2/contributor-guide/contributing_dod.html
-[devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.html
+[DOCUMENTATION_TEMPLATE]: https://github.com/magento/devdocs/blob/master/.github/DOCUMENTATION_TEMPLATE.md
 [existing issues]: https://github.com/magento/magento2-functional-testing-framework/issues?q=is%3Aopen+is%3Aissue
 [existing PRs]: https://github.com/magento/magento2-functional-testing-framework/pulls?q=is%3Aopen+is%3Apr
 [GitHub documentation]: https://help.github.com/articles/syncing-a-fork
@@ -171,4 +171,4 @@ Label| Description
 [Magento Contributor Agreement]: http://www.magento.com/legaldocuments/mca
 [MFTF repository]: https://github.com/magento/magento2-functional-testing-framework
 [open new issue]: https://github.com/magento/magento2-functional-testing-framework/issues/new
-[Travis CI]: https://travis-ci.com/magento/magento2-functional-testing-framework/pull_requests
+[Travis CI]: https://travis-ci.com/magento/magento2-functional-testing-framework/pull_requests

.github/DOCUMENTATION_TEMPLATE.md

@@ -0,0 +1,26 @@
+# How to contribute to MFTF docs
+
+We welcome contributions to the MFTF documentation, which is kept within the `docs/` folder in this repository.
+This page describes the submitting process and serves as a template for a properly written content.
+The contribution workflow is the same as submitting code.
+
+1. Create a branch from the `develop` branch in the [MFTF repository][].
+1. Make edits/additions/deletions as needed. Read the [Basic Template][] for tips on how to write with Markdown.
+1. Submit your pull request to the `develop` branch.
+
+Once submitted, a member of the documentation team will review and merge it.
+We will inform you if it needs any additional processing.
+
+The documentation in this repository is used as the source for the [MFTF documentation][].
+Any changes to the table of contents will need to be made through a separate pull request in the regular [Magento Developer documentation repository][].
+
+Read more about how to [Contribute to Magento Devdocs][].
+
+<!-- For readability, we abstract the link URLS to the bottom of the page. The extra set of square brackets denotes it is a link, rather than plain brackets. >
+
+<!-- Link Definitions -->
+[Magento Developer documentation repository]: https://github.com/magento/devdocs/blob/master/_data/toc/mftf.yml
+[MFTF repository]: https://github.com/magento/magento2-functional-testing-framework
+[Contribute to Magento Devdocs]: https://github.com/magento/devdocs/blob/master/.github/CONTRIBUTING.md
+[MFTF documentation]: https://devdocs.magento.com/mftf/docs/introduction.html
+[Basic Template]: https://devdocs.magento.com/guides/v2.3/contributor-guide/templates/basic_template.html

.gitignore

@@ -16,4 +16,6 @@ dev/tests/functional/MFTF.suite.yml
 dev/tests/functional/_output
 dev/mftf.log
 dev/tests/mftf.log
-dev/tests/docs/*
+dev/tests/docs/*
+dev/tests/_output
+dev/tests/functional.suite.yml

CHANGELOG.md

@@ -1,6 +1,96 @@
 Magento Functional Testing Framework Changelog
 ================================================
 
+2.5.1
+-----
+
+### Fixes
+* Fixed missing `use` statement in the generate:suite command
+
+### GitHub Issues
+* [#471](https://github.com/magento/magento2-functional-testing-framework/issues/471) -- PHP Fatal error: MftfApplicationConfig not found in GenerateSuiteCommand
+
+2.5.0
+-----
+* Traceability
+    * Allure output has been enhanced to contain new test artifacts created and used per MFTF step:
+        * `makeScreenshot` will contain an attachment under its Allure step.
+        * `seeInCurrentUrl` and all other `Url` asserts now contain an attachment with the expects vs actual comparison.
+        * `createData` and all other `Data` actions now contain attachments with `Request Body` and `Response Body`.
+* Modularity
+    * Added a new `mftf run:manifest` command to run testManifest files generated by `generate:tests`.
+        * See DevDocs for details
+    * `mftf generate/run:test` commands now implicitly generates the `suite` the test exists in.
+        * If a test exists in multiple suites, it will generate it in all suite contexts.
+    * `mftf run:test <testName>` will now only run the exact test provided, regardless of what is generated.
+* Maintainability
+    * Added an `--allow-skipped` flag that allows MFTF to ignore the `<skip>` annotation. This was added to the following commands:
+        * `generate:test`
+        * `run:test`
+        * `run:group`
+        * `run:failed`
+* Customizability
+    * `<entity>` defined in data.xml can now reference other `<entity>` directly.
+        * See DevDocs for details
+    * Added vault as an alternative credential storage.
+        * See DevDocs for details
+
+### Fixes
+* Fixed an issue where `grab` action variables were not substituting correctly when used as an element parameter.
+* Framework will not throw a descriptive error when referencing a `$persisted.field$` that does not exist.
+* MFTF test materials that `extends=""` itself will no longer cause infinite recursion.
+* Fixed an issue where a test could not reference a `$data.field$` whose casing was modified by the API that it used.
+* Fixed an issue with the default `functional.suite.yml` where it was incompatible with `symfony/yaml 4.0.0`.
+* Improved test generation performance via class refactors (`~10%` faster).
+
+### GitHub Issues/Pull requests:
+* [#377](https://github.com/magento/magento2-functional-testing-framework/pull/377) -- Non-API operations fixes
+
+2.4.5
+-----
+### Fixes
+* Fixed an issue where `.credentials` was required when using `<createData>` actions with field overrides.
+
+2.4.4
+-----
+### Fixes
+* Fixed an issue where `_CREDS` could not be resolved when used in a suite.
+
+2.4.3
+-----
+* Customizability
+    * Use of `_CREDS` has been extended to `<magentoCLI>` and `<createData>` actions
+* Traceability
+    * A Test step is now generated and injected in the allure report when a test is reported as `BROKEN`. 
+
+### Fixes
+* `static-checks` command now properly returns `1` if any static check failed.
+* MFTF Console Printer class correctly utilizes `--steps` and other flags passed directly to `codecept commands`.
+* `*source` actions correctly print when using `userInput` or `html` attributes.
+* XML Comments should no longer throw an error in parsing when used outside `test/actionGroup`
+
+### GitHub Issues/Pull requests:
+* [#703](https://github.com/magento/magento2-functional-testing-framework/pull/403) -- SMALL_CHANGE: Minor change suggested
+
+2.4.2
+-----
+* Traceability
+    * Test action `stepKey`s are now included in both console output and Allure report.
+    * XML Comments are now automatically converted into a `<comment>` action.
+
+### Fixes
+* Moved `epfremme/swagger-php` dependency to `suggests` block due to a conflict with Magento extensions.
+
+2.4.1
+-----
+* Traceability
+    * XSD Schema validation is now enabled by default in `generate:tests`, `run:test`, `run:failed`, `run:group`
+    * `--debug` option for the above commands has been updated to include different debug levels
+        * See DevDocs for details
+
+### Fixes
+* Fixed an issue where `skipReadiness` attribute would cause false XSD Schema validation errors.
+
 2.4.0
 -----
 ### Enhancements

bin/mftf

@@ -29,7 +29,7 @@ try {
 try {
     $application = new Symfony\Component\Console\Application();
     $application->setName('Magento Functional Testing Framework CLI');
-    $application->setVersion('2.4.0');
+    $application->setVersion('2.5.1');
     /** @var \Magento\FunctionalTestingFramework\Console\CommandListInterface $commandList */
     $commandList = new \Magento\FunctionalTestingFramework\Console\CommandList;
     foreach ($commandList->getCommands() as $command) {

composer.json

@@ -2,19 +2,21 @@
     "name": "magento/magento2-functional-testing-framework",
     "description": "Magento2 Functional Testing Framework",
     "type": "library",
-    "version": "2.4.0",
+    "version": "2.5.1",
     "license": "AGPL-3.0",
     "keywords": ["magento", "automation", "functional", "testing"],
     "config": {
         "sort-packages": true
     },
     "require": {
         "php": "7.0.2||7.0.4||~7.0.6||~7.1.0||~7.2.0||~7.3.0",
-        "allure-framework/allure-codeception": "~1.3.0",
         "ext-curl": "*",
+        "allure-framework/allure-codeception": "~1.3.0",
         "codeception/codeception": "~2.3.4 || ~2.4.0 ",
+        "composer/composer": "^1.6",
         "consolidation/robo": "^1.0.0",
-        "epfremme/swagger-php": "^2.0",
+        "csharpru/vault-php": "~3.5.3",
+        "csharpru/vault-php-guzzle6-transport": "^2.0",
         "flow/jsonpath": ">0.2",
         "fzaninotto/faker": "^1.6",
         "monolog/monolog": "^1.0",
@@ -36,6 +38,9 @@
         "php-coveralls/php-coveralls": "^1.0",
         "symfony/stopwatch": "~3.4.6"
     },
+    "suggest": {
+        "epfremme/swagger-php": "^2.0"
+    },
     "autoload": {
         "files": ["src/Magento/FunctionalTestingFramework/_bootstrap.php"],
         "psr-4": {