Test documentation added

Author: anatoliykmetyuk

docs/docs/contributing/testing.md

@@ -54,6 +54,54 @@ You can also run all paths of classes of a certain name:
 > testOnly *.TreeTransformerTest
 ```
 
+### Testing with checkfiles
+Some tests support checking the output of the run or the compilation against a checkfile. A checkfile is a file in which the expected output of the compilation or run is defined. A test against a checkfile fails if the actual output mismatches the expected output.
+
+Currently, the `neg` (compilation must fail for the test to succeed) and `run` support the checkfiles. `neg`'s checkfiles contain an expected error output during compilation. `run`'s checkfiles contain an expected run output of the successfully compiled program.
+
+Absence of a checkfile is **not** a condition for the test failure. E.g. if a `neg` test fails with the expected number of errors and there is no checkfile for it, the test still passes.
+
+Checkfiles are located in the same directories as the tests they check, have the same name as these tests with the extension `*.check`. E.g. if you have a test named `tests/neg/foo.scala`, you can create a checkfile for it named `tests/neg/foo.check`. And if you have a test composed of several files in a single directory, e.g. `tests/neg/manyScalaFiles`, the checkfile will be `tests/neg/manyScalaFiles.check`.
+
+If the actual output mismatches the expected output, the test framework will dump the actual output in the file `*.check.out` and fail the test suite. It will also output the instructions to quickly replace the expected output with the actual output, in the following format:
+
+```
+Test output dumped in: tests/playground/neg/Sample.check.out
+  See diff of the checkfile
+    > diff tests/playground/neg/Sample.check tests/playground/neg/Sample.check.out
+  Replace checkfile with current output output
+    > mv tests/playground/neg/Sample.check.out tests/playground/neg/Sample.check
+```
+
+To create a checkfile for a test, you can do one of the following:
+
+- Create a dummy checkfile with a random content, run the test, and, when it fails, use the `mv` command reported by the test to replace the dummy checkfile with the actual output.
+- Manually compile the file you are testing with `dotc` and copy-paste whatever console output the compiler produces to the checkfile.
+
+The latter way will work in most cases, but not all of them. This is because the test framework runs `dotc` with a number of flags, e.g.:
+
+```
+dotc -classpath /Users/anatolii/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.12.8.jar:/Users/anatolii/.ivy2/cache/org.scala-lang.modules/scala-xml_2.12/bundles/scala-xml_2.12-1.1.0.jar:/Users/anatolii/Projects/dotty/dotty/library/../out/bootstrap/dotty-library-bootstrapped/scala-0.15/dotty-library_0.15-0.15.0-bin-SNAPSHOT.jar  -Yno-deep-subtypes -Yno-double-bindings -Yforce-sbt-phases -Xverify-signatures -pagewidth  120 -color:never -Ycheck:all  tests/playground/neg/Sample.scala
+```
+
+These can sometimes influence the console output. If it happens, you can obtain the precise command used for the compilation from the reproduction instructions. The reproduction instructions are logged by the test framework for each failed test, you can find them by locating the following output of the failed test run:
+
+
+```
+================================================================================
+Test Report
+================================================================================
+
+1 suites passed, 1 failed, 2 total
+    tests/playground/neg/Sample.scala failed
+
+
+--------------------------------------------------------------------------------
+Note - reproduction instructions have been dumped to log file:
+    /Users/anatolii/Projects/dotty/dotty/testlogs/tests-2019-05-14/tests-2019-05-14-T14-56-49.log
+--------------------------------------------------------------------------------
+```
+
 ## Integration tests
 These tests are Scala source files expected to compile with Dotty (pos tests),
 along with their expected output (run tests) or errors (neg tests).