[feature-req] watch tests and auto-reload on change (like chimp); Don't shutdown the browser

[dheerajbhaskar]

What are you trying to achieve?

I'm trying to get a realtime feedback for quicker test development cycle. Just like chimp where:

1. Browser stays open after tests are run
2. Tests are re-run when the test files change
3. auto manage selenium server and browser drivers
4. Auto installs dependencies

What do you get instead?

No realtime feedback on test writing

1. Browser shuts down after each run
2. Have to manually re-run tests
3. Need to start selenium manually
4. Need to install helpers manually (even though it's mentioned during init)

Provide console output if related. Use --verbose mode for more details.

# paste output here

Provide test source code if related

// paste test

Details

• CodeceptJS version: 0.5.1
• NodeJS Version: v6.10.2
• Operating System: win 10 pro x64
• Protractor || WebDriverIO || Nightmare version (if related): webdriverio: 4.6.2
• Configuration file:

codecept.json

{
  "tests": "./feature/*.js",
  "timeout": 10000,
  "output": "./output",
  "helpers": {
    "WebDriverIO": {
      "url": "http://localhost",
      "browser": "chrome",
      "restart": true,
      "windowSize": "maximize",
      "timeouts": {
        "script": 60000,
        "page load": 10000,
        "implicit" : 5000
      }
    }
  },
  "include": {
    "I": "./custom-steps/steps_file.js"
  },
  "bootstrap": false,
  "mocha": {},
  "name": "shaadi-try"
}

# paste suite config here

[tamascsaba]

It will be nice 👍

[leojh]

Would integrating with jest give this behavior out of the box?

[dheerajbhaskar]

Based on their website and Github, jest seems to be for unit testing JavaScript, not Integration tests like selenium and codeceptjs.

These days I'm using webdriverio which also has a watch mode (not like chimp though; it kills and reloads the browser after each change). However, I've found webdriverio to be vastly more stable than chimp. Chimp document says something and it just doesn't work when you use it, I used to pull my hair out while writing tests which wouldn't work as expected (like $, $$ operators are unreliable/not working, etc) . Then realized it was chimp. Tried fixing it, but couldn't get it to work reliably.

Long story short, if this issue were to be pursued by anyone I'd suggest integrating webdriverio rather than chimp

[leojh]

I see. I have something similar to that now but using jest instead of webdriverio's runner. It works fine, but was looking to get additional features. Im going to take a second look at using their built in runner tho.

[spidgorny]

In the meantime you can try nodemon codeceptjs run, maybe?

[lydia-schow]

Almost! This works:

nodemon --exec "codeceptjs run"

Typescript:

I'm using typescript, but nodemon only watches javascript files by default, so I needed to specify a file extension with -e:

nodemon --exec "codeceptjs run" -e ts

Watch a directory

I also wanted to only --watch the test directory

nodemon --exec "codeceptjs run" -e ts --watch src/end-to-end-tests

NPM Script

That's a lot to type, so I made it an npm script, and now I can run end-to-end tests with:

yarn test:e2e

Or, npm run test:e2e if you're using npm.

[VaasRamsay]

@DavertMik Any thought on this? I can see that this can be achieved by nodemon or similar tools but it would mean the init and loadTests would be done over and again. This means a heavy restart duration for larger projects. It'd not be a problem if watch mode was available out-of-the-box. What do you think?

[VaasRamsay]

@DavertMik any plans for this in the roadmap?

[adelin-b]

I bumped in this issue a few time now. On the ui it shows that tests have been reloaded with a green toast. However it only reload the .feature and not the steps. The dev flow take a big hit.