Travis-CI: Use APT to install GEOS

[micahcochran]

Install libgeos using apt would probably save almost 4 minutes of build time. There is also a newer version of GEOS already packaged.

In .travis.yaml replace the compiling of libgeos with this:

addons:
  apt:
    packages:
    - libgeos-3.3.8

Here are the docs for APT on Travis.

I'm putting this here until I can get PR #234 working and merged.

[jenshnielsen]

This sounds excellent. We may need to have the package whitelisted if not already done. See https://github.com/travis-ci/apt-package-whitelist

[micahcochran]

There are currently 3 versions in the whitelist: libgeos-3.2.2, libgeos-3.3.3, and libgeos-3.3.8.

[jenshnielsen]

Great

[WeatherGod]

Is it possible to use the APT version in some build and the bundled version
in other builds so that we can make sure that things work both ways?

On Tue, Dec 8, 2015 at 9:27 AM, Jens Hedegaard Nielsen <
notifications@github.com> wrote:

Great

—
Reply to this email directly or view it on GitHub
#235 (comment).

[micahcochran]

@WeatherGod It is possible. That makes the configuration slightly more complicated. I think the way to do that would be to have a GEOS_CURRENT flag that build version that compiles the most recent libgeos release, 3.5.0 for Python 2.7 & 3.5. GEOS_DIR will have to be to set to the just compiled directory (it defaults to /usr/local).

The APT installed version will skip the compilation step when GEOS_CURRENT== True.

Maybe GEOS_CURRENT should be GEOS or GEOS_VERSION, I'm not that familiar with Travis-CI configuration. I'm sure this approach can be refined.

[micahcochran]

In order to accomplish this it might not be a bad idea if test.py has a function (software_versions)that prints out all of the software versions of installed modules. It could be logged as informational an printed if the tests are ran verbosely (-v).

It might not be a bad way to diagnose future problems.

This could be in test.py or it could be a part of the standard library.

[pelson]

FWIW, a lot of projects have had huge success using conda as the tool for package management on travis. That would allow full control of a GEOS / Proj.4 / ... version matrix, and is very quick to debug and diagnose. If you wanted to go down that route and have any questions, I'd be happy to help.

Cheers,

[micahcochran]

Searching on Anaconda.org for "GEOS" did not yield any results. Wheels can be used, if there are any. conda might be a good way to test multiple versions of pyproj. However, I've seen bug reports about some versions of pyproj 1.9.4 not being compiled right on conda. I'm willing to explore.

[pelson]

Interesting. https://anaconda.org/search?q=geos yields a number of results for me.

However, I've seen bug reports about some versions of pyproj 1.9.4 not being compiled right on conda

I believe that may be the case. @ocefpaf maintains a wonderful set of conda distributions over at However, I've seen bug reports about some versions of pyproj 1.9.4 not being compiled right on conda which are provided through the IOOS anaconda channel - if they aren't compiled right, you can go ahead and fix them at https://github.com/ioos/conda-recipes and it is built upon merge. Pretty slick way of keeping dependencies up to date etc.

I've a lot of experience setting up travis with conda, and have reduced the problem to a pretty small .travis.yml file:

# The language in this case has no bearing - we are going to be making use of conda for a
# python distribution for the scientific python stack.
language: python

sudo: false

env:
    global:
       - CONDA_INSTALL_LOCN="${HOME}/conda"

    matrix:
        - PYTHON=2.7
        - PYTHON=3.4

install:
    - mkdir -p ${HOME}/cache/pkgs
    - "[ ! -f ${HOME}/cache/miniconda.sh ] && wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ${HOME}/cache/miniconda.sh || :"
    - bash ${HOME}/cache/miniconda.sh -b -p ${CONDA_INSTALL_LOCN} && export PATH=${CONDA_INSTALL_LOCN}/bin:$PATH

    # Re-use the pacakges in the cache, and download any new ones into that location.
    - rm -rf ${CONDA_INSTALL_LOCN}/pkgs && ln -s ${HOME}/cache/pkgs ${CONDA_INSTALL_LOCN}/pkgs

    # Now do the things we need to do to install it.
    - conda install --file requirements.txt nose python=${PYTHON} --yes --quiet
    - python setup.py install
    - mkdir not_the_source_dir && cd not_the_source_dir

script:
    - nosetests conda_testenv


# We store the files that are downloaded from continuum.io, but not the environments that are created.
cache:
    directories:
      - $HOME/cache
before_cache:
  # Remove all untarred directories.
  - find $CONDA_INSTALL_LOCN/pkgs/ -mindepth 1 -maxdepth 1 -type d -exec rm -r {} \;

It is by no means without its warts, but it is at least well field tested. It is easy enough to add the IOOS channel to the conda install call, and you would have a functional testbed I believe.

[micahcochran]

Interesting. https://anaconda.org/search?q=geos yields a number of results for me.

Thanks. It looks like any uppercase letters in the name throws off the search engine.

[pelson]

It looks like any uppercase letters in the name throws off the search engine.

So it does. I'd suggest raising that at https://github.com/ContinuumIO/anaconda-issues.

[ocefpaf]

@micahcochran and @pelson if I recall it correctly there are two issues with pyproj on the default anaconda channel.

1. Pyproj is compiled against an independent proj.4 build and not the bundled (and tweaked) proj.4 source. That is OK for most of pyproj features, but some EPSG transformations will fail. (That might be a problem here.)
2. It is broken on Windows. (I believe that is not a problem here.)

Both issues are fixed in the pyproj provided in the IOOS channel.

PS: I am trying to make the IOOS channel a place to get niche specific packages and not a place that has alternatives to the default channel. I believe that the latter should be hosted in a community channel like conda-forge. I am submitting our version of pyproj there today.

[pelson]

I am submitting our version of pyproj there today.

👍 to that @ocefpaf.

[micahcochran]

@ocefpaf pyproj has a patch for PROJ.4 for the hammer projection to provide an inverse. (Which hammer is one of the projections supported by basemap.) As discussed here pyproj4/pyproj#19 . That is the only difference.

[ocefpaf]

Thanks @micahcochran that is good to know. However, the default channel is missing the EPSG data and compiling using the bundled source "make it work." Maybe I should dig deeper into that to ensure that the default channel also has that in the future.