Skip to content

Commit 7924421

Browse files
Test documentation added
1 parent a7cee09 commit 7924421

File tree

1 file changed

+24
-0
lines changed

1 file changed

+24
-0
lines changed

docs/docs/contributing/testing.md

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,30 @@ You can also run all paths of classes of a certain name:
5454
> testOnly *.TreeTransformerTest
5555
```
5656

57+
### Testing with checkfiles
58+
Some tests support checking the output of the run or the compilation against a checkfile. A checkfile is a file in which the expected output of the compilation or run is defined. A test against a checkfile fails if the actual output mismatches the expected output.
59+
60+
Currently, the `run` and `neg` (compilation must fail for the test to succeed) tests support the checkfiles. `run`'s checkfiles contain an expected run output of the successfully compiled program. `neg`'s checkfiles contain an expected error output during compilation.
61+
62+
Absence of a checkfile is **not** a condition for the test failure. E.g. if a `neg` test fails with the expected number of errors and there is no checkfile for it, the test still passes.
63+
64+
Checkfiles are located in the same directories as the tests they check, have the same name as these tests with the extension `*.check`. E.g. if you have a test named `tests/neg/foo.scala`, you can create a checkfile for it named `tests/neg/foo.check`. And if you have a test composed of several files in a single directory, e.g. `tests/neg/manyScalaFiles`, the checkfile will be `tests/neg/manyScalaFiles.check`.
65+
66+
If the actual output mismatches the expected output, the test framework will dump the actual output in the file `*.check.out` and fail the test suite. It will also output the instructions to quickly replace the expected output with the actual output, in the following format:
67+
68+
```
69+
Test output dumped in: tests/playground/neg/Sample.check.out
70+
See diff of the checkfile
71+
> diff tests/playground/neg/Sample.check tests/playground/neg/Sample.check.out
72+
Replace checkfile with current output output
73+
> mv tests/playground/neg/Sample.check.out tests/playground/neg/Sample.check
74+
```
75+
76+
To create a checkfile for a test, you can do one of the following:
77+
78+
- Create a dummy checkfile with a random content, run the test, and, when it fails, use the `mv` command reported by the test to replace the dummy checkfile with the actual output.
79+
- Manually compile the file you are testing with `dotc` and copy-paste whatever console output the compiler produces to the checkfile.
80+
5781
## Integration tests
5882
These tests are Scala source files expected to compile with Dotty (pos tests),
5983
along with their expected output (run tests) or errors (neg tests).

0 commit comments

Comments
 (0)