Skip to content

Better image testing docs #67

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 4, 2015
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 69 additions & 17 deletions test/image/README.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,80 @@
# plotly.js image testing

Test plotly.js with the Plotly Image-Server docker container.
Test plotly.js with Plotly's image testing docker container.

Requirements:
- `docker` | [installation guidelines](http://docs.docker.com/engine/installation/)
- `docker-machine` (for Mac and Windows users only) | [installation guidelines](https://docs.docker.com/machine/install-machine/)
- `docker-compose` | [installation guidelines](https://docs.docker.com/compose/install/)

### Run the container with `docker compose`
### Step 0: Start the docker machine (Mac and Windows users only)

Boot up docker machine (named `default`):

```bash
docker-machine start default
```

Set up the docker environment for `docker-compose`:

```bash
eval $(docker-machine env default)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexcjohnson this should do the trick.

Reference: docker/compose#2180 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alexcjohnson @cldougl what version of docker-compose are you using?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker-compose version: 1.5.1

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yup, the eval did it for me. (I used it with quotes, as mentioned within the output of docker-machine env default itself)
Image tests work locally for me now! Well almost, some intermittent gl3d errors (gl3d_marker-color.png failed twice in a row, gl3d_mirror-ticks.png failed one of two times)

Does it save the images it makes somewhere when they fail?

Version: 1.5.1, just like @cldougl (installed today)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it save the images it makes somewhere when they fail?

Yes, look into build/test_images and `build/test_images_diff``.

Or even better, run npm run start-image_viewer to view the baseline, generated image, diff and mock side by side.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A good shell function to add to your rc file if you have lots of docker-machines (I have like 8 that I use depending) is this.

```

the above evaluates the output of `docker-machine env default`.


### Step 1: Run the testing container

Plotly.js uses `docker-compose` to ease the creation/stopping/deletion of the testing docker container.

Plotly.js uses `docker-compose` to ease the creation/stopping/deletion of testing docker container,
please refer [installing docker-compose](https://docs.docker.com/compose/install/) for installation.
Inside your `plotly.js` directory, run

```bash
$ docker-compose up -d
docker-compose up -d
```

In the `docker-compose.yml` file, `latest` is the latest Plotly Image-Server docker container version
as listed on [hub.docker.com](https://hub.docker.com/r/plotly/imageserver/tags/) and
`imagetest` is the name of the docker container.

`imagetest` is the name of the docker container. The `-d` flag tells docker to start the containers in the background and leave them running.

### Run the tests
### Step 2: Run the image tests

Inside your `plotly.js` directory, run

```bash
npm run test-image
```

### SSH into docker
if some tests fail, compare their outputs using `npm run start-image_viewer`.

### Step 2b: Make a new baseline image

Inside your `plotly.js` directory, run

```bash
npm run baseline -- mock.json
```

where `mock.json` is the name of a `{"data": [], "layout": {}}` json file found in [`test/image/mocks/`](https://github.com/plotly/plotly.js/tree/master/test/image/mocks). The `"data"` and `"layout"` field are passed to `Plotly.plot` to produce an image saved in [`test/image/baslines`](https://github.com/plotly/plotly.js/tree/master/test/image/baselines).

### Step 3: Stop your testing container

Once done testing, inside your `plotly.js` directory, run

```bash
docker-compose stop
```

Mac and Windows user should also kill their docker-machine (named `default`) once done testing:

```bash
docker-machine kill default
```

### Docker tricks

##### SSH into docker container

```bash
ssh -p 2022 root@localhost # with password `root`
Expand Down Expand Up @@ -58,36 +106,40 @@ ssh-keygen -f "${HOME}/.ssh/known_hosts" -R [localhost]:2022

to remove host information.

##### List docker machines

```bash
docker-machine ls
```

### List all images
##### List all images

```bash
docker images
```

### List all containers
##### List all containers

```bash
docker ps -a
```

whereas `docker ps` lists only the started containers.


### Stop your testing container
##### Remove your testing container

Inside your `plotly.js` directory, run

```bash
docker-compose stop
docker-compose rm -f
```

### Remove your testing container
##### Remove your docker machine

Inside your `plotly.js` directory, run
If named `default`:

```bash
docker-compose rm -f
docker-machine kill default
```

For more comprehensive information about docker, please refer to the [docker docs](http://docs.docker.com/).