Standalone has issues validating tests in 2.3.0-beta32

[lazyguru]

I am trying to build our own suite to run tests. I am testing using MFTF standalone per the directions here https://devdocs.magento.com/mftf/2.3/getting-started.html#set-up-a-standalone-mftf. When I try to generate a single test or our suite, I get errors from other (unrelated) tests.

See Slack thread for more info: https://magentocommeng.slack.com/archives/C82DDUSNP/p1542748941043300

Preconditions

1. Using standalone MFTF per https://devdocs.magento.com/mftf/2.3/getting-started.html#set-up-a-standalone-mftf
2. Have 2.3.0-beta32 installed
3. Have a custom suite (or at minimum custom test) in a custom module

Steps to reproduce

1. Run bin/mftf generate:suite MySuite or bin/mftf generate:test MySingleTest

Expected result

1. Suite or Test should generate (or throw an error only about specific tests)

Actual result

The following files all fail validation for different reasons. I only captured the actual errors from the first one. I manually deleted the files at each run to try to get the generation of just our suite to run. I also ran the generation with the debug flag (-d) to get this output

1. ee23/vendor/magento/module-multiple-wishlist/Test/Mftf/Section/StorefrontCreateMultipleWishlistSection.xml

In ActionMergeUtil.php line 278:

  ee23/vendor/magento/module-multiple-wishlist/Test/Mftf/Section/StorefrontCreateMultipleWishlistSection.xml

  Invalid Document

  Element 'element', attribute 'type': [facet 'enumeration'] The value 'image' is not an element of the set {'text', 'textarea', 'input', 'button', 'checkbox', 'radio', 'checkboxset', 'radioset', 'date', 'file', 'select', 'multiselect', 'wysiwyg', 'iframe', 'block'}.

  Line: 12

  Element 'element', attribute 'type': 'image' is not a valid value of the atomic type 'uiElementType'.

  Line: 12

  .

  Exception occurred parsing action at StepKey "ApiVersionNA"

generate:suite [-r|--remove] [--] <suites> (<suites>)...

2. ee23/vendor/magento/module-customer-custom-attributes/Test/Mftf/Section/CustomerAccountSection.xml
3. ee23/vendor/magento/module-theme/Test/Mftf/Section/StorefrontFooterSection.xml
4. ee23/vendor/magento/module-configurable-product/Test/Mftf/Test/ConfigurableProductPriceAdditionalStoreViewTest.xml
5. ee23/vendor/magento/module-catalog/Test/Mftf/Test/AdminAddImageToWYSIWYGProductTest.xml

[okolesnyk]

Hi @lazyguru
Thank you for reporting this issue.
We created internal ticket MQE-1367 to track and resolve this issue.

[tomreece]

@okolesnyk I see that the internal ticket you mention (https://jira.corp.magento.com/browse/MQE-1367) is CLOSED as DONE.

It's not immediately clear to me how that issue solves this problem, but if you feel that it does then please close this github issue or I will close it by the end of the week. Thanks!

[lbajsarowicz]

@lazyguru I'm trying to find out if the issue you reported still occurs.

Could you verify that with latest MFTF version and share the update?

I'm asking because if the failure still occurs, I can address that next weekend.

[lazyguru]

I'm not sure. We don't try running via standalone anymore and just use the built-in version of MFTF.

[tomreece]

@lazyguru If you still have problems, please create a new ticket.