pip error on Macbook pro (m1 pro)

[CaffreyR]

Hi! I am working on Macbook pro with basemap, but I met some issues. When I try to pip install basemap, there is something wrong with this. But I have already brew install geos, what's wrong? Thanks!

[molinav]

Hi @CaffreyR! Did you set the GEOS_DIR environment variable? Since your traceback says that the file "geos_c.h" is not found, I would assume that the _geoslib module cannot be compiled because of not finding the GEOS headers in your system.

[ronaldbradford]

I'm getting the same problem, and I'm setting GEOS_DIR.
Brew installs geos=3.10.2, this comment in basemap suggests installing an older version #533

[molinav]

Hi @ronaldbradford, thanks for the feedback. Could you confirm me if you can find a "geos_c.h" file in your system and the value that you are assigning to GEOS_DIR?

[ronaldbradford]

I was using GEOS_DIR=/usr/local/Cellar/geos/3.10.2

I've uninstalled and building form source right now to see if I can confirm it fixes my issuel

[ronaldbradford]

I have installed geos 3.8.2 from source https://libgeos.org/usage/download/

$ ls -l $GEOS_DIR/include/geos_c.h
-rw-r--r--  1 rbradfor  staff  97001 Apr  7 17:26 /usr/local/include/geos_c.h
$ echo $GEOS_DIR
/usr/local

$ pip3 install basemap
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Collecting basemap
  Using cached basemap-1.3.2.zip (155 kB)
  ...
  Building wheels for collected packages: basemap
  Building wheel for basemap (pyproject.toml) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/opt/python@3.9/bin/python3.9 /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/hp/l11pqpjn721d4cjpw1qjzl_c0000gn/T/tmpjrhbdnvf
       cwd: /private/var/folders/hp/l11pqpjn721d4cjpw1qjzl_c0000gn/T/pip-install-0vpt9rzd/basemap_ce944475e96546c5846addd29810dfff
  Complete output (32 lines):
  setup.py:52: RuntimeWarning: Cannot find GEOS library and/or headers in standard locations ('/usr/local'). Please install the corresponding packages using your software management system or set the environment variable GEOS_DIR to point to the location where GEOS is installed (for example, if 'geos_c.h' is in '/usr/local/include' and 'libgeos_c' is in '/usr/local/lib', then you need to set GEOS_DIR to '/usr/local'
    warnings.warn(" ".join([
  running bdist_wheel
  running build
  running build_py
  ...

[ronaldbradford]

FWIW 'm also getting the same error with

$ pip3 install basemap==1.3.1
$ pip3 install basemap==1.3.0

[molinav]

From your traceback I would infer that there is no problem with finding the header file but with finding the library itself.

The problematic line could be this one in the setup file:

basemap/packages/basemap/setup.py

Line 45 in f9c9dab

libname = "{0}*.{1}*".format(libcode, libext)

The setup file tries to find the library assuming that the library filename follows the pattern "libgeos_c*.so*" and is placed inside $GEOS_DIR/lib, $GEOS_DIR/lib/x86_64-linux-gnu or $GEOS_DIR/lib64. Is this correct for MacOS? If not, then the setup file would need an update.

[ronaldbradford]

I have installed geos 3.9.2 now from source https://libgeos.org/usage/download/
I Installed basemap from source via https://github.com/matplotlib/basemap instructions
Debugging setup.py is beyond my knowledge level. I know it's late there, please let me know what I can try.

$  ls -l $GEOS_DIR/include/geos_c.h
-rw-r--r--  1 rbradfor  staff  79397 Apr  7 17:53 /usr/local/include/geos_c.h
$ python -m pip install .
DEPRECATION: Configuring installation scheme with distutils config files is deprecated and will no longer work in the near future. If you are using a Homebrew or Linuxbrew Python, please see discussion at https://github.com/Homebrew/homebrew-core/issues/76621
Processing /Users/rbradfor/tmp/basemap/packages/basemap
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy<1.23,>=1.21 in /usr/local/lib/python3.9/site-packages (from basemap==1.3.2+dev) (1.22.2)
Requirement already satisfied: basemap-data<1.4,>=1.3.2 in /usr/local/lib/python3.9/site-packages (from basemap==1.3.2+dev) (1.3.2)
Requirement already satisfied: matplotlib<3.6,>=1.5 in /usr/local/lib/python3.9/site-packages (from basemap==1.3.2+dev) (3.5.1)
Collecting pyshp<2.2,>=1.2
  Using cached pyshp-2.1.3-py3-none-any.whl
Requirement already satisfied: pyproj<3.4.0,>=1.9.3 in /usr/local/lib/python3.9/site-packages (from basemap==1.3.2+dev) (3.3.0)
Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (9.0.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (1.3.2)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (3.0.7)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (21.3)
Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (4.29.1)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (0.11.0)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.9/site-packages (from matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (2.8.2)
Requirement already satisfied: certifi in /usr/local/lib/python3.9/site-packages (from pyproj<3.4.0,>=1.9.3->basemap==1.3.2+dev) (2021.10.8)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib<3.6,>=1.5->basemap==1.3.2+dev) (1.16.0)
Building wheels for collected packages: basemap
  Building wheel for basemap (pyproject.toml) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/local/opt/python@3.9/bin/python3.9 /usr/local/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/hp/l11pqpjn721d4cjpw1qjzl_c0000gn/T/tmp8iq42jtw
       cwd: /Users/rbradfor/tmp/basemap/packages/basemap
  Complete output (19 lines):
  setup.py:52: RuntimeWarning: Cannot find GEOS library and/or headers in standard locations ('/usr/local'). Please install the corresponding packages using your software management system or set the environment variable GEOS_DIR to point to the location where GEOS is installed (for example, if 'geos_c.h' is in '/usr/local/include' and 'libgeos_c' is in '/usr/local/lib', then you need to set GEOS_DIR to '/usr/local'
    warnings.warn(" ".join([
  running bdist_wheel
  running build
  running build_py
  running build_ext
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -I/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include -I/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/_geoslib.c -o build/temp.macosx-10.15-x86_64-3.9/src/_geoslib.o
  In file included from src/_geoslib.c:685:
  In file included from /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include/numpy/arrayobject.h:5:
  In file included from /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include/numpy/ndarrayobject.h:12:
  In file included from /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include/numpy/ndarraytypes.h:1960:
  /usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with "          "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
  #warning "Using deprecated NumPy API, disable it with " \
   ^
  src/_geoslib.c:686:10: fatal error: 'geos_c.h' file not found
  #include "geos_c.h"
           ^~~~~~~~~~
  1 warning and 1 error generated.
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for basemap
Failed to build basemap
ERROR: Could not build wheels for basemap, which is required to install pyproject.toml-based projects
WARNING: You are using pip version 21.3.1; however, version 22.0.4 is available.
You should consider upgrading via the '/usr/local/opt/python@3.9/bin/python3.9 -m pip install --upgrade pip' command.

[ronaldbradford]

$ ls -l $GEOS_DIR/lib/libgeos_c*
-rwxr-xr-x  1 rbradfor  staff   226536 Apr  7 17:35 /usr/local/lib/libgeos_c.1.13.4.dylib
-rwxr-xr-x  1 rbradfor  staff   233112 Apr  7 17:49 /usr/local/lib/libgeos_c.1.14.3.dylib
-rwxr-xr-x  1 rbradfor  staff  1407648 Apr  7 17:58 /usr/local/lib/libgeos_c.1.9.1.dylib
lrwxr-xr-x  1 rbradfor  staff       21 Apr  7 17:58 /usr/local/lib/libgeos_c.1.dylib -> libgeos_c.1.9.1.dylib
lrwxr-xr-x  1 rbradfor  staff       17 Apr  7 17:35 /usr/local/lib/libgeos_c.dylib -> libgeos_c.1.dylib
$ uname -a
Darwin atlantis.lan 19.6.0 Darwin Kernel Version 19.6.0: Sun Nov 14 19:58:51 PST 2021; root:xnu-6153.141.50~1/RELEASE_X86_64 x86_64

Let me interactively look at ~L45 and see.

[ronaldbradford]

>>> import os
>>> os.name
'posix'
>>> libfiles = []
>>> libdirs = ["bin", "lib", "lib/x86_64-linux-gnu", "lib64"]
>>> libext = "dll" if os.name == "nt" else "so"
>>> libcode = "{0}geos_c".format("" if os.name == "nt" else "lib")
>>> libname = "{0}*.{1}*".format(libcode, libext)
>>> for libdir in libdirs:
...   libfiles.extend(glob.glob(os.path.join(prefix, libdir, libname)))
>>> print(libfiles)
[]
>>> print("{},{},{}".format(libext,libcode,libname))
so,libgeos_c,libgeos_c*.so*

So yes, there is no *.so file on MacOS

[ronaldbradford]

4121440

[molinav]

This traceback shows much more information. Something is going wrong when setting the compilation flags internally, because the clang call is missing a -I$GEOS_DIR/include (with $GEOS_DIR expanded to your value). This is supposed to be done by the setup.py but it is clear that it is not doing its job in MacOS.

Can you check if setting the environment variable CFLAGS="-I$GEOS_DIR/include" in advance lets you go further with the installation?

[molinav]

4121440

You got it. This commit done by me is causing at least one of the problems. Before, only the presence of the header file was checked. I wanted to ensure that the library was actually there too, but only considered the extensions for Windows (.dll) and for manylinux (.so).

I will need to update this. But still I am confused, because in your other traceback the first check seems to pass but later the compilation of _geoslib fails due to the missing clang header argument.

What is clear is that I should create a GitHub Action to build MacOS wheels, because with them this error would have appeared fast.

It is a bit late for me now but I will try to take a look to this either tomorrow afternoon or this weekend.

[CaffreyR]

Hi @CaffreyR! Did you set the GEOS_DIR environment variable? Since your traceback says that the file "geos_c.h" is not found, I would assume that the _geoslib module cannot be compiled because of not finding the GEOS headers in your system.

I'm getting the same problem, and I'm setting GEOS_DIR. Brew installs geos=3.10.2, this comment in basemap suggests installing an older version #533

Yep, I have set the dir in .bash_profile too. As you can see above. @molinav

And does an older version work? @ronaldbradford

[molinav]

@CaffreyR You need to define GEOS_DIR as an independent environment variable instead of adding the GEOS directory to PATH, like so:

export GEOS_DIR="/opt/homebrew/Cellar/geos/3.10.2"

Still, this won't be enough because the setup file needs a correction as shown by @ronaldbradford above. Here the GEOS version does not play a role: the recommendation about the version limit for GEOS is due to a problem only when drawing the Antarctic coast.