Create hardware continuous testing environment

[ed7coyne]

We should have a testing rig setup that will:

• Monitor our github repos for new changes (definitely merges, maybe even PRs)
• Pull locally
• Run all examples against a local esp module
• Scrape serial output for expected values
• Report results, maybe via email or maybe post to PRs (ala travis), maybe create an issue on failure.

[igrr]

FYI, there is some plumbing for automated testing available here:
https://github.com/esp8266/Arduino/tree/master/tests/device
make test_something/test_something.ino in that directory compiles the test, flashes it to the board and runs all test cases.
Feel free to use, abuse, and comment.

edit:
JUnit-compatible XML test results are placed into .build/test_something/test_results.xml for every test which is run. Currently I use this in combination with Jenkins.

[ed7coyne]

Awesome! Thanks for the pointer, looks like that makefile is doing a good chunk of what we want. When we get around to trying this we will definitely let you know how it works for us.

[ed7coyne]

I am going to take this on, I think we need this running soon to start catching some of these problems before users.