Skip to content

Add documentation for new parallel testing suite #2153

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Mar 31, 2017
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions doc-tool/resources/css/dottydoc.css
Original file line number Diff line number Diff line change
Expand Up @@ -334,3 +334,23 @@ blockquote {
color: #777;
border-left: 0.25em solid #ddd;
}

aside {
padding: 15px;
margin: 10px 0;
}

aside.warning {
border-left: 3px solid #d62c2c;
background-color: #ffe4e4;
}

aside.notice {
border-left: 3px solid #4c97e4;
background-color: #e4ebff;
}

aside.success {
border-left: 3px solid #36bf1d;
background-color: #ebfddd;
}
89 changes: 89 additions & 0 deletions docs/docs/contributing/testing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
---
layout: doc-page
title: Testing in Dotty
---

<aside class="warning">
This page should be updated as soon as scala-partest is removed
</aside>

Running all tests in Dotty is as simple as:

```bash
$ sbt test
```

There are currently several forms of tests in Dotty. These can be split into
two categories:

## Unit tests
These tests can be found in `<sub-project>/test` and are used to check
functionality of specific parts of the codebase in isolation e.g: parsing,
scanning and message errors.

Running a single unit test class from sbt is as simple as:

```bash
> testOnly absolute.path.to.TestClass
```

You can further restrict the executed tests to a subset of `TestClass` methods
as follows:

```bash
> testOnly absolute.path.to.TestClass -- *methodName
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain the * here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

> testOnlyt path.toTestClass -- -z subStringOfMethod
> testOnlyt path.toTestClass -- -t exactMethodName

From what I gather the *methodName is equivalent to -z methodName

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I include this instead?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I not sure it's better, but it might be a bit confusing to have both .* and * in the such similar places

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided to merge this so that @abeln can base his docs update on it.

I'll get to the docs again once I've fixed the deadlock situation.

```

## Integration tests
These tests are Scala source files expected to compile with Dotty (pos tests),
along with their expected output (run tests) or errors (neg tests).

All of these tests are contained in the `./tests/*` directories.

## scala-partest
Historically these tests needed a structure which was generated by running the
unit tests, and then that structure was in turn used by
[scala-partest](http://github.com/scala/scala-partest) to run compilation tests
in parallel.

This test suite can still be used (and is currently a part of the CI to check
that it has the same outcome as the new test suite). It is invoked from sbt by
running one of the following commands:

```bash
> partest-only-no-bootstrap
> partest-only
> partest
```

- `partest-only-no-bootstrap` will only run the integration tests
- `partest-only` will bootstrap the compiler and run the integration tests
- `partest` will bootstrap the compiler, run the unit tests and then the
integration tests

## dotty parallel test suite
The new test suite will soon become the standard integration test runner. It
has several advantages over the old implementation:

- integrates with JUnit, without the need for setup
- reuses the same VM for compilation
- allows filtering of tests
- runs much faster (almost 2x)

Currently to run these tests you need to invoke from sbt:

```bash
> testOnly dotty.tools.dotc.CompilationTests
```

This might be aliased in the future. It is also possible to run tests filtered
by using:

```bash
> filterTest .*i2147.scala
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about using testOnly for both by class name and by regex filtering?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that would be cool. Please implement :trollface:

```

This will run both the test `./tests/pos/i2147.scala` and
`./tests/partest-test/i2147.scala` since both of these match the given regular
expression. This also means that you could run `filterTest .*` to run all
integration tests.
16 changes: 13 additions & 3 deletions docs/docs/contributing/workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,11 +57,21 @@ $ sbt
To test a specific test tests/x/y.scala (for example tests/pos/t210.scala):

```bash
> partest-only-no-bootstrap --show-diff --verbose tests/partest-generated/x/y.scala
> filterTest .*pos/t210.scala
```

Currently this will re-run some unit tests and do some preprocessing because of
the way partest has been set up.
The filterTest task takes a regular expression as its argument. For example,
you could run a negative and a positive test with:

```bash
> filterTest (.*pos/t697.scala)|(.*neg/i2101.scala)
```

or if they have the same name, the equivalent can be achieved with:

```bash
> filterTest .*/i2101.scala
```

## Inspecting Trees with Type Stealer ##

Expand Down
9 changes: 5 additions & 4 deletions docs/docs/internals/higher-kinded-v2.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ layout: doc-page
title: "Higher-Kinded Types in Dotty"
---

**This page is out of date and preserved for posterity. Please see [Implementing
Higher-Kinded Types in
Dotty](http://guillaume.martres.me/publications/dotty-hk.pdf) for a more up to
date version**
<aside class="warning">
This page is out of date and preserved for posterity. Please see
<a href="http://guillaume.martres.me/publications/dotty-hk.pdf">
Implementing Higher-Kinded Types in Dotty</a> for a more up to date version
</aside>

Higher-Kinded Types in Dotty V2
===============================
Expand Down
2 changes: 2 additions & 0 deletions docs/sidebar.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ sidebar:
url: docs/contributing/intellij-idea.html
- title: Workflow
url: docs/contributing/workflow.html
- title: Testing
url: docs/contributing/testing.html
- title: Internals
subsection:
- title: Backend
Expand Down