bug(cli-ui): `yarn run test:e2e` does not run `yarn run test:e2e:dev`

[Kocal]

Version

3.0.1

Reproduction link

https://github.com/vuejs/vue-cli

Node and OS info

Node 9.11.2 / Yarn 1.7.0 / Debian 9.4 (stretch)

Steps to reproduce

• git clone https://github.com/vuejs/vue-cli.git
• cd vue-cli
• yarn
• cd packages/cli-ui
• yarn run test:e2e

What is expected?

When url http://localhost:4040/graphql is responding, it should automatically run yarn run test:e2e:dev.

What is actually happening?

It just run an apollo server (script apollo:run:test), url is responding, but it does not do anything after.

➜  cli-ui git:(chore/docs-cli-ui) yarn run test:e2e                  
yarn run v1.7.0
$ yarn run test:clear && start-server-and-test apollo:run:test http://localhost:4040 test:e2e:dev
$ rimraf ../../test/cli-ui-test && rimraf ./live-test
starting server using command "npm run apollo:run:test"
and when url "http://localhost:4040" is responding
running tests using command "test:e2e:dev"

> @vue/cli-ui@3.0.1 apollo:run:test /home/kocal/Dev/vue-cli/packages/@vue/cli-ui
> cross-env VUE_CLI_UI_TEST=true VUE_APP_GRAPHQL_PORT=4040 VUE_APP_CLI_UI_URL=ws://localhost:4040/graphql VUE_CLI_IPC=vue-cli-test vue-cli-service apollo:watch

$ cross-env VUE_CLI_DEBUG=true VUE_CLI_IPC=vue-cli-dev vue-cli-service apollo:run --delay
✔️  GraphQL Server is running on http://localhost:4040/graphql

---

Initial discussion: #2281 (comment)

[Kocal]

This also happens on:

• Debian 9.5 (Stretch), with Node.js 9.11.2 and Yarn 1.9.4
• Windows 10, with Node.js 8.11.4 and Yarn 1.9.4

By the way, we expect URL http://localhost:4040 to respond before running tests, but it returns a 404:

{ ~ }  » curl -I http://localhost:4040                                                      ~ 130
HTTP/1.1 404 Not Found
X-Powered-By: Express
Content-Security-Policy: default-src 'self'
X-Content-Type-Options: nosniff
Content-Type: text/html; charset=utf-8
Content-Length: 140
Date: Wed, 22 Aug 2018 13:03:18 GMT
Connection: keep-alive

Maybe the real issue is here? http://localhost:4040/graphql successfully display a GraphQL Playground interface.

[Kocal]

This was definitely related, I opened a PR #2323.

[Akryum]

Can't reproduce on linux (elementary OS loki) with either node 8.11.3/npm 5.6.0 or node 10.8.0/npm 6.2.0

[Kocal]

That's really strange... 🤔
What happens if you run DEBUG=start-server-and-test yarn run test:e2e?

[Kocal]

@Akryum I'm sorry to insist, but I really want to know what DEBUG=start-server-and-test yarn run test:e2e outputs to you.
I don't understand how it can works since http://localhost:4040 returns a 404.

Thank you

[Akryum]

akryum@akryum-XPS-13-9360:~/Projets/vue-cli/packages/@vue/cli-ui$ DEBUG=start-server-and-test yarn run test:e2e
yarn run v1.9.4
$ yarn run test:clear && start-server-and-test apollo:run:test http://localhost:4040 test:e2e:dev
$ rimraf ../../test/cli-ui-test && rimraf ./live-test
starting server using command "npm run apollo:run:test"
and when url "http://localhost:4040" is responding
running tests using command "test:e2e:dev"
  start-server-and-test starting server, verbose mode? true +0ms
  start-server-and-test starting waitOn http://localhost:4040 +6ms

> @vue/cli-ui@3.0.1 apollo:run:test /home/akryum/Projets/vue-cli/packages/@vue/cli-ui
> cross-env VUE_CLI_UI_TEST=true VUE_APP_GRAPHQL_PORT=4040 VUE_APP_CLI_UI_URL=ws://localhost:4040/graphql VUE_CLI_IPC=vue-cli-test vue-cli-service apollo:watch

$ cross-env VUE_CLI_DEBUG=true VUE_CLI_IPC=vue-cli-dev vue-cli-service apollo:run --delay
child next [ { 'http://localhost:4040': -999 } ]
wait-on(17831) waiting for: http://localhost:4040
child complete
✔️  GraphQL Server is running on http://localhost:4040/graphql
child next []
child complete
child next [ { 'http://localhost:4040': 200 } ]
child complete
child next []
stabilized
wait-on(17831) exiting successfully found all: http://localhost:4040
  start-server-and-test waitOn finished successfully +8s
  start-server-and-test running test script command test:e2e:dev +7ms

> @vue/cli-ui@3.0.1 test:e2e:dev /home/akryum/Projets/vue-cli/packages/@vue/cli-ui
> cross-env VUE_APP_CLI_UI_URL=ws://localhost:4040/graphql vue-cli-service test:e2e --mode development

 INFO  Starting e2e tests...

[Kocal]

What the fuck. 🤨
Well, I will close this issue and the associated PR since it seems to works both on your laptop and on CircleCi. 🤷‍♂️

[Akryum]

Apollo Server 2 introduced an health-check path so I'll update the script to use it: https://www.apollographql.com/docs/apollo-server/whats-new.html#Health-checks

[Kocal]

Nice