ARM64: AttributeError: type object 'asyncpg.protocol.protocol.Codec' has no attribute '__reduce_cython__'

[kamikaze]

Getting same (#501 and #565) issue with Python 3.8.3 and:

asyncpg==0.20.1
asyncpgsa==0.26.3
psycopg2==2.8.5

on arm64

src/my_app/db/__init__.py:1: in <module>
    from asyncpgsa import PG
../.venv38/lib/python3.8/site-packages/asyncpgsa/__init__.py:1: in <module>
    from .pool import create_pool
../.venv38/lib/python3.8/site-packages/asyncpgsa/pool.py:3: in <module>
    import asyncpg
../.venv38/lib/python3.8/site-packages/asyncpg/__init__.py:8: in <module>
    from .connection import connect, Connection  # NOQA
../.venv38/lib/python3.8/site-packages/asyncpg/connection.py:19: in <module>
    from . import connect_utils
../.venv38/lib/python3.8/site-packages/asyncpg/connect_utils.py:27: in <module>
    from . import protocol
../.venv38/lib/python3.8/site-packages/asyncpg/protocol/__init__.py:8: in <module>
    from .protocol import Protocol, Record, NO_TIMEOUT  # NOQA
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

>   ???
E   AttributeError: type object 'asyncpg.protocol.protocol.Codec' has no attribute '__reduce_cython__'

[kamikaze]

$ python
Python 3.8.3 (default, May 29 2020, 07:06:49) 
[GCC 10.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from asyncpg import PG
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/asyncpg/__init__.py", line 8, in <module>
    from .connection import connect, Connection  # NOQA
  File "/home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/asyncpg/connection.py", line 19, in <module>
    from . import connect_utils
  File "/home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/asyncpg/connect_utils.py", line 27, in <module>
    from . import protocol
  File "/home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/asyncpg/protocol/__init__.py", line 8, in <module>
    from .protocol import Protocol, Record, NO_TIMEOUT  # NOQA
  File "asyncpg/protocol/protocol.pyx", line 1, in init asyncpg.protocol.protocol
ImportError: /home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/asyncpg/pgproto/pgproto.cpython-38-aarch64-linux-gnu.so: undefined symbol: uuid_to_hex

[Gelbpunkt]

@kamikaze The commit that fixes this isn't in the pypi release yet. Try using git+https://github.com/MagicStack/asyncpg.git instead of specifying a pypi version. That should use the latest code as long as you have git installed and worked fine on aarch64 for me.

[kamikaze]

@kamikaze The commit that fixes this isn't in the pypi release yet. Try using git+https://github.com/MagicStack/asyncpg.git instead of specifying a pypi version. That should use the latest code as long as you have git installed and worked fine on aarch64 for me.

Collecting asyncpg-0.20.2@ https://github.com/MagicStack/asyncpg/tarball/master#egg=asyncpg-0.20.2
  Downloading https://github.com/MagicStack/asyncpg/tarball/master
     \ 201 kB 253 kB/s
    ERROR: Command errored out with exit status 1:
     command: /home/kamikaze/projects/proj/.venv38/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-uoh6mqt_/asyncpg-0.20.2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-uoh6mqt_/asyncpg-0.20.2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-kc4k163m
         cwd: /tmp/pip-install-uoh6mqt_/asyncpg-0.20.2/
    Complete output (44 lines):
    running egg_info
    creating /tmp/pip-pip-egg-info-kc4k163m/asyncpg.egg-info
    writing /tmp/pip-pip-egg-info-kc4k163m/asyncpg.egg-info/PKG-INFO
    writing dependency_links to /tmp/pip-pip-egg-info-kc4k163m/asyncpg.egg-info/dependency_links.txt
    writing requirements to /tmp/pip-pip-egg-info-kc4k163m/asyncpg.egg-info/requires.txt
    writing top-level names to /tmp/pip-pip-egg-info-kc4k163m/asyncpg.egg-info/top_level.txt
    writing manifest file '/tmp/pip-pip-egg-info-kc4k163m/asyncpg.egg-info/SOURCES.txt'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-uoh6mqt_/asyncpg-0.20.2/setup.py", line 250, in <module>
        setuptools.setup(
      File "/home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/setuptools/__init__.py", line 161, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 297, in run
        self.find_sources()
      File "/home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 304, in find_sources
        mm.run()
      File "/home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 535, in run
        self.add_defaults()
      File "/home/kamikaze/projects/proj/.venv38/lib/python3.8/site-packages/setuptools/command/egg_info.py", line 571, in add_defaults
        sdist.add_defaults(self)
      File "/usr/lib/python3.8/distutils/command/sdist.py", line 228, in add_defaults
        self._add_defaults_ext()
      File "/usr/lib/python3.8/distutils/command/sdist.py", line 311, in _add_defaults_ext
        build_ext = self.get_finalized_command('build_ext')
      File "/usr/lib/python3.8/distutils/cmd.py", line 299, in get_finalized_command
        cmd_obj.ensure_finalized()
      File "/usr/lib/python3.8/distutils/cmd.py", line 107, in ensure_finalized
        self.finalize_options()
      File "/tmp/pip-install-uoh6mqt_/asyncpg-0.20.2/setup.py", line 234, in finalize_options
        self.distribution.ext_modules[:] = cythonize(
      File "/tmp/pip-install-uoh6mqt_/asyncpg-0.20.2/.eggs/Cython-0.29.14-py3.8-linux-aarch64.egg/Cython/Build/Dependencies.py", line 959, in cythonize
        module_list, module_metadata = create_extension_list(
      File "/tmp/pip-install-uoh6mqt_/asyncpg-0.20.2/.eggs/Cython-0.29.14-py3.8-linux-aarch64.egg/Cython/Build/Dependencies.py", line 810, in create_extension_list
        for file in nonempty(sorted(extended_iglob(filepattern)), "'%s' doesn't match any files" % filepattern):
      File "/tmp/pip-install-uoh6mqt_/asyncpg-0.20.2/.eggs/Cython-0.29.14-py3.8-linux-aarch64.egg/Cython/Build/Dependencies.py", line 109, in nonempty
        raise ValueError(error_msg)
    ValueError: 'asyncpg/pgproto/pgproto.pyx' doesn't match any files
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

[kamikaze]

it seems it doesn't pull submodules... nah...

[Gelbpunkt]

Oh yeah - I usually clone, init submodules and build a wheel to reuse it as the ARM machines I use are very weak and take many minutes to compile it. Have you tried that yet?

[kamikaze]

Oh yeah - I usually clone, init submodules and build a wheel to reuse it as the ARM machines I use are very weak and take many minutes to compile it. Have you tried that yet?

yep, it works. but is this project alive? I would like to get it packed and released for automated stuff. Manual installation is a no go in real world :)

[elprans]

yep, it works. but is this project alive

Alive and well.

I would like to get it packed and released for automated stuff

We use Travis for testing and releases, feel free to add a build target to CI.

[Gelbpunkt]

@elprans I think he was referring to releasing the current codebase as a new version to include the updated pgproto in upstream. I would be happy to work on the Travis target for aarch64 if you wouldn't mind

[elprans]

I would be happy to work on the Travis target for aarch64 if you wouldn't mind

That would be welcome. Thanks!

[Gelbpunkt]

@elprans #595 should fix it 😄

[kamikaze]

please...

[Gelbpunkt]

@kamikaze there are now aarch64 wheels on pypi for version 0.21 so I think this is resolved 👍