Skip to content

Package infrastructure cleanup #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 23 commits into from
Jan 26, 2020
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
81aa03a
Moved setup metadata to setup.cfg
astrofrog Jan 17, 2020
14d8fa9
Improve tox.ini configuration
astrofrog Jan 17, 2020
168ae50
Reduce required run-time dependencies
astrofrog Jan 17, 2020
2edda86
More improvements to tox set-up
astrofrog Jan 17, 2020
e424b41
Updated CI configuration to use tox
astrofrog Jan 17, 2020
c3974b3
Remove AppVeyor badge
astrofrog Jan 17, 2020
1d92f3d
Check that the plugin doesn't stop pytest from running if Matplotlib …
astrofrog Jan 17, 2020
5c4b759
Pass DISPLAY environment variable to tox
astrofrog Jan 17, 2020
e107724
Fix code style issues
astrofrog Jan 17, 2020
5f41a2d
Simplify Travis test matrix
astrofrog Jan 17, 2020
c19f5bf
Fixes to tox and CI
astrofrog Jan 17, 2020
99a899d
Fix CI issues
astrofrog Jan 17, 2020
de262f6
Updated baseline image for test_base_style
astrofrog Jan 17, 2020
4f860ec
Fix tests on Windows
astrofrog Jan 17, 2020
7be0270
Updated README.rst and remove pillow as a dependency
astrofrog Jan 17, 2020
b26ee6b
Remove dependency on pillow and instead delegate to Matplotlib's imread
astrofrog Jan 17, 2020
8d5fcbc
Added note about install_requires
astrofrog Jan 17, 2020
a8f434a
Fix default tolerance
astrofrog Jan 17, 2020
7edc17e
Fix code style
astrofrog Jan 17, 2020
0165446
Explicitly specify testpaths and reorder test matrix to have slowest …
astrofrog Jan 17, 2020
d944a53
Add back Matplotlib as a required dependency
astrofrog Jan 20, 2020
182a65a
Drop support for Python 3.5 and earlier, as well as Matplotlib 1.5
astrofrog Jan 23, 2020
ff26395
Remove old comment
astrofrog Jan 24, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 44 additions & 34 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,46 +1,56 @@
language: c
language: python

# Setting sudo to false opts in to Travis-CI container-based builds.
sudo: false

os:
- linux
- osx

env:
global:
- SETUP_XVFB=True
- CONDA_DEPENDENCIES="pytest matplotlib nose coverage freetype=2.5.5"
- PIP_DEPENDENCIES="pytest-cov coveralls"
matrix:
- PYTHON_VERSION=3.4 MATPLOTLIB_VERSION=1.5
- PYTHON_VERSION=2.7 MATPLOTLIB_VERSION=1.5
- PYTHON_VERSION=3.5 MATPLOTLIB_VERSION=1.5
- PYTHON_VERSION=2.7 MATPLOTLIB_VERSION=2.0
- PYTHON_VERSION=3.5 MATPLOTLIB_VERSION=2.0
- PYTHON_VERSION=3.6 MATPLOTLIB_VERSION=2.0

# The following build is meant to check that dependencies get set up correctly, but currently
# the image tests fail, which should be investigated.
# - PYTHON_VERSION=3.5 CONDA_DEPENDENCIES="coverage freetype libpng"
- TOXENV='test'
- TOXARGS='-v'
- TOXPOSARGS=''

matrix:
include:
# Test the oldest and newest configuration on Mac and Windows
- os: osx
language: c
env: PYTHON_VERSION=2.7 TOXENV=py27-test-mpl15
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we just drop it now?

- os: osx
language: c
env: PYTHON_VERSION=3.8 TOXENV=py38-test-mpl31
- os: windows
language: c
env: PYTHON_VERSION=2.7 TOXENV=py27-test-mpl15
- os: windows
language: c
env: PYTHON_VERSION=3.8 TOXENV=py38-test-mpl31
# Test all configurations on Linux
- python: 2.7
env: TOXENV=py27-test-mpl15
- python: 3.5
env: TOXENV=py35-test-mpl20
- python: 3.6
env: TOXENV=py36-test-mpl21
- python: 3.6
env: TOXENV=py36-test-mpl22
- python: 3.7
env: TOXENV=py37-test-mpl30
- python: 3.8
env: TOXENV=py38-test-mpl31
- python: 3.8
env: TOXENV=py38-test-minimal
- python: 3.8
env: TOXENV=codestyle

install:

# We use the ci-helpers package from the Astropy project to set up conda
# with any requested dependencies above.
- git clone git://github.com/astropy/ci-helpers.git
- source ci-helpers/travis/setup_conda_$TRAVIS_OS_NAME.sh

# Need to use develop instead of install to make sure coverage works
- python setup.py develop
- if [[ $TRAVIS_OS_NAME != linux ]]; then
git clone git://github.com/astropy/ci-helpers.git;
source ci-helpers/travis/setup_conda.sh;
fi

script:
- python -c 'import pytest_mpl.plugin'
- pytest -vv --mpl --cov pytest_mpl tests
# Make sure that the tests run ok even without the --mpl option (we close
# figures anyway in this case)
- pytest -vv --cov pytest_mpl --cov-append tests
- python setup.py check --restructuredtext
- pip install tox
- tox $TOXARGS -- $TOXPOSARGS

after_success:
- coveralls
- coveralls
59 changes: 18 additions & 41 deletions README.rst
Original file line number Diff line number Diff line change
@@ -1,16 +1,11 @@
|Travis Build Status| |AppVeyor Build status| |Coveralls coverage|
|Travis Build Status| |Coveralls coverage|

About
-----

This is a plugin to facilitate image comparison for
`Matplotlib <http://www.matplotlib.org>`__ figures in pytest.

Matplotlib includes a number of test utilities and decorators, but these
are geared towards the `nose <http://nose.readthedocs.org/>`__ testing
framework. Pytest-mpl makes it easy to compare figures produced by tests
to reference images when using `pytest <http://pytest.org>`__.

For each figure to test, the reference image is subtracted from the
generated image, and the RMS of the residual is compared to a
user-specified tolerance. If the residual is too large, the test will
Expand All @@ -23,23 +18,17 @@ section below.
Installing
----------

This plugin is compatible with Python 2.6, 2.7, and 3.3 and later, and
requires `pytest <http://pytest.org>`__,
`matplotlib <http://www.matplotlib.org>`__ and
`nose <http://nose.readthedocs.org/>`__ to be installed (nose is
required by Matplotlib).

To install, you can do:
This plugin is compatible with Python 2.7, and 3.5 and later, and
requires `pytest <http://pytest.org>`__ and
`matplotlib <http://www.matplotlib.org>` to be installed.

::
To install, you can do::

pip install pytest-mpl

You can check that the plugin is registered with pytest by doing:

::
You can check that the plugin is registered with pytest by doing::

py.test --version
pytest --version

which will show a list of plugins:

Expand Down Expand Up @@ -71,24 +60,20 @@ function returns a Matplotlib figure (or any figure object that has a

To generate the baseline images, run the tests with the
``--mpl-generate-path`` option with the name of the directory where the
generated images should be placed:

::
generated images should be placed::

py.test --mpl-generate-path=baseline
pytest --mpl-generate-path=baseline

If the directory does not exist, it will be created. The directory will
be interpreted as being relative to where you are running ``py.test``.
be interpreted as being relative to where you are running ``pytest``.
Once you are happy with the generated images, you should move them to a
sub-directory called ``baseline`` relative to the test files (this name
is configurable, see below). You can also generate the baseline images
directly in the right directory.

You can then run the tests simply with:
You can then run the tests simply with::

::

py.test --mpl
pytest --mpl

and the tests will pass if the images are the same. If you omit the
``--mpl`` option, the tests will run but will only check that the code
Expand Down Expand Up @@ -145,11 +130,9 @@ a comma-separated list of URLs (real commas in the URL should be encoded
as ``%2C``).

Finally, you can also set a custom baseline directory globally when
running tests by running ``py.test`` with:

::
running tests by running ``pytest`` with::

py.test --mpl --mpl-baseline-path=baseline_images
pytest --mpl --mpl-baseline-path=baseline_images

This directory will be interpreted as being relative to where the tests
are run. In addition, if both this option and the ``baseline_dir``
Expand Down Expand Up @@ -190,9 +173,7 @@ Test failure example

If the images produced by the tests are correct, then the test will
pass, but if they are not, the test will fail with a message similar to
the following:

::
the following::

E Exception: Error: Image files did not match.
E RMS Value: 142.2287807767823
Expand Down Expand Up @@ -226,7 +207,7 @@ By default, the expected, actual and difference files are written to a
temporary directory with a non-deterministic path. If you want to instead
write them to a specific directory, you can use::

py.test --mpl --mpl-results-path=results
pytest --mpl --mpl-results-path=results

The ``results`` directory will then contain one sub-directory per test, and each
sub-directory will contain the three files mentioned above. If you are using a
Expand All @@ -242,20 +223,16 @@ Running the tests for pytest-mpl
--------------------------------

If you are contributing some changes and want to run the tests, first
install the latest version of the plugin then do:

::
install the latest version of the plugin then do::

cd tests
py.test --mpl
pytest --mpl

The reason for having to install the plugin first is to ensure that the
plugin is correctly loaded as part of the test suite.

.. |Travis Build Status| image:: https://travis-ci.org/matplotlib/pytest-mpl.svg?branch=master
:target: https://travis-ci.org/matplotlib/pytest-mpl
.. |AppVeyor Build status| image:: https://ci.appveyor.com/api/projects/status/mf7hs44scg5mvcyo?svg=true
:target: https://ci.appveyor.com/project/astrofrog/pytest-mpl
.. |Coveralls coverage| image:: https://coveralls.io/repos/matplotlib/pytest-mpl/badge.svg
:target: https://coveralls.io/r/matplotlib/pytest-mpl
.. |expected| image:: images/baseline-coords_overlay_auto_coord_meta.png
Expand Down
37 changes: 0 additions & 37 deletions appveyor.yml

This file was deleted.

32 changes: 19 additions & 13 deletions pytest_mpl/plugin.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def _download_file(baseline, filename):
try:
u = urlopen(base_url + filename)
content = u.read()
except Exception as exc:
except Exception:
warnings.warn('Downloading {0} failed'.format(base_url + filename))
else:
break
Expand Down Expand Up @@ -104,7 +104,9 @@ def pytest_addoption(parser):

def pytest_configure(config):

config.addinivalue_line('markers', "mpl_image_compare: Compares matplotlib figures against a baseline image")
config.addinivalue_line('markers',
"mpl_image_compare: Compares matplotlib figures "
"against a baseline image")

if config.getoption("--mpl") or config.getoption("--mpl-generate-path") is not None:

Expand Down Expand Up @@ -192,8 +194,8 @@ def pytest_runtest_setup(self, item):
if compare is None:
return

from PIL import Image
import matplotlib
from matplotlib.image import imread
import matplotlib.pyplot as plt
from matplotlib.testing.compare import compare_images
try:
Expand Down Expand Up @@ -269,23 +271,28 @@ def item_function_wrapper(*args, **kwargs):
if baseline_remote:
baseline_image_ref = _download_file(baseline_dir, filename)
else:
baseline_image_ref = os.path.abspath(os.path.join(os.path.dirname(item.fspath.strpath), baseline_dir, filename))
baseline_image_ref = os.path.abspath(os.path.join(
os.path.dirname(item.fspath.strpath), baseline_dir, filename))

if not os.path.exists(baseline_image_ref):
pytest.fail("Image file not found for comparison test in: "
"\n\t{baseline_dir}"
"\n(This is expected for new tests.)\nGenerated Image: "
"\n\t{test}".format(baseline_dir=baseline_dir, test=test_image), pytrace=False)
"\n\t{test}".format(baseline_dir=baseline_dir,
test=test_image),
pytrace=False)

# distutils may put the baseline images in non-accessible places,
# copy to our tmpdir to be sure to keep them in case of failure
baseline_image = os.path.abspath(os.path.join(result_dir, 'baseline-' + filename))
baseline_image = os.path.abspath(os.path.join(result_dir,
'baseline-' + filename))
shutil.copyfile(baseline_image_ref, baseline_image)

# Compare image size ourselves since the Matplotlib exception is a bit cryptic in this case
# and doesn't show the filenames
expected_shape = Image.open(baseline_image).size
actual_shape = Image.open(test_image).size
# Compare image size ourselves since the Matplotlib
# exception is a bit cryptic in this case and doesn't show
# the filenames
expected_shape = imread(baseline_image).shape[:2]
actual_shape = imread(test_image).shape[:2]
if expected_shape != actual_shape:
error = SHAPE_MISMATCH_ERROR.format(expected_path=baseline_image,
expected_shape=expected_shape,
Expand All @@ -305,7 +312,8 @@ def item_function_wrapper(*args, **kwargs):
if not os.path.exists(self.generate_dir):
os.makedirs(self.generate_dir)

fig.savefig(os.path.abspath(os.path.join(self.generate_dir, filename)), **savefig_kwargs)
fig.savefig(os.path.abspath(os.path.join(self.generate_dir, filename)),
**savefig_kwargs)
close_mpl_figure(fig)
pytest.skip("Skipping test, since generating data")

Expand All @@ -331,8 +339,6 @@ def pytest_runtest_setup(self, item):
if compare is None:
return

import matplotlib.pyplot as plt

original = item.function

@wraps(item.function)
Expand Down
41 changes: 41 additions & 0 deletions setup.cfg
Original file line number Diff line number Diff line change
@@ -1,2 +1,43 @@
[metadata]
license_file = LICENSE
name = pytest-mpl
version = 0.11
url = https://github.com/matplotlib/pytest-mpl
author = Thomas Robitaille
author_email = thomas.robitaille@gmail.com
classifiers =
Development Status :: 4 - Beta
Framework :: Pytest
Intended Audience :: Developers
Topic :: Software Development :: Testing
Topic :: Scientific/Engineering :: Visualization
Programming Language :: Python
Programming Language :: Python :: 2
Programming Language :: Python :: 3
Operating System :: OS Independent
License :: OSI Approved :: BSD License
license = BSD
description = pytest plugin to help with testing figures output from Matplotlib
long_description = file: README.rst

[options]
zip_safe = True
packages = find:
# Note that we don't include Matplotlib as a required dependency since it is only
# required if the plugin is actually used, in which case the package using this
# will already need to have Matplotlib declared as a dependency. We want to avoid
# installing Matplotlib unecessarily.
install_requires =
pytest

[options.entry_points]
pytest11 =
pytest_mpl = pytest_mpl.plugin

[options.extras_require]
test =
pytest-cov
matplotlib

[tool:pytest]
testpaths = "tests"
Loading