"ImportError: cannot import name 'Mapping' from 'collections'" with Python 3.10

[simonkrenger]

Environment information (required)

Diagnostics output

--- check: autoidentify
INFO: diagnose_tensorboard.py version e43767ef2b648d0d5d57c00f38ccbd38390e38da

--- check: general
INFO: sys.version_info: sys.version_info(major=3, minor=10, micro=1, releaselevel='final', serial=0)
INFO: os.name: posix
INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='fedora-ml.example.com', release='5.15.10-200.fc35.x86_64', version='#1 SMP Fri Dec 17 14:46:39 UTC 2021', machine='x86_64')
INFO: sys.getwindowsversion(): N/A

--- check: package_management
INFO: has conda-meta: False
INFO: $VIRTUAL_ENV: None

--- check: installed_packages
INFO: installed: tensorboard==2.7.0
INFO: installed: tensorflow==2.8.0rc0
INFO: installed: tf-estimator-nightly==2.8.0.dev2021122109
INFO: installed: tensorboard-data-server==0.6.1

--- check: tensorboard_python_version
INFO: tensorboard.version.VERSION: '2.7.0'

--- check: tensorflow_python_version
2021-12-24 22:11:13.556272: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2021-12-24 22:11:13.556660: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
INFO: tensorflow.__version__: '2.8.0-rc0'
INFO: tensorflow.__git_version__: 'v1.12.1-69099-g0d00ab2fd15'

--- check: tensorboard_data_server_version
INFO: data server binary: '/home/simon/.local/lib/python3.10/site-packages/tensorboard_data_server/bin/server'
INFO: data server binary version: b'rustboard 0.6.1'

--- check: tensorboard_binary_path
INFO: which tensorboard: b'/home/simon/.local/bin/tensorboard\n'

--- check: addrinfos
socket.has_ipv6 = True
socket.AF_UNSPEC = <AddressFamily.AF_UNSPEC: 0>
socket.SOCK_STREAM = <SocketKind.SOCK_STREAM: 1>
socket.AI_ADDRCONFIG = <AddressInfo.AI_ADDRCONFIG: 32>
socket.AI_PASSIVE = <AddressInfo.AI_PASSIVE: 1>
Loopback flags: <AddressInfo.AI_ADDRCONFIG: 32>
Loopback infos: [(<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::1', 0, 0, 0)), (<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('127.0.0.1', 0))]
Wildcard flags: <AddressInfo.AI_PASSIVE: 1>
Wildcard infos: [(<AddressFamily.AF_INET: 2>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('0.0.0.0', 0)), (<AddressFamily.AF_INET6: 10>, <SocketKind.SOCK_STREAM: 1>, 6, '', ('::', 0, 0, 0))]

--- check: readable_fqdn
INFO: socket.getfqdn(): 'fedora-ml.example.com'

--- check: stat_tensorboardinfo
INFO: directory: /tmp/.tensorboard-info
INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=2071, st_dev=38, st_nlink=2, st_uid=1000, st_gid=1000, st_size=60, st_atime=1640379357, st_mtime=1640379357, st_ctime=1640379357)
INFO: mode: 0o40777

--- check: source_trees_without_genfiles
INFO: tensorboard_roots (1): ['/home/simon/.local/lib/python3.10/site-packages']; bad_roots (0): []

--- check: full_pip_freeze
INFO: pip freeze --all:
absl-py==1.0.0
argcomplete==1.12.3
argon2-cffi==21.3.0
argon2-cffi-bindings==21.2.0
astunparse==1.6.3
attrs==21.2.0
backcall==0.2.0
Beaker==1.10.0
beautifulsoup4==4.9.3
bleach==4.1.0
blivet==3.4.2
blivet-gui==2.3.0
Brlapi==0.8.2
cachetools==4.2.4
cffi==1.14.6
chardet==4.0.0
charset-normalizer==2.0.4
chrome-gnome-shell==0.0.0
click==8.0.1
cryptography==3.4.7
cupshelpers==1.0
cycler==0.11.0
dasbus==1.6
dbus-python==1.2.18
debugpy==1.5.1
decorator==5.1.0
defusedxml==0.7.1
distro==1.6.0
entrypoints==0.3
fedora-third-party==0.8
flatbuffers==2.0
fonttools==4.28.5
fros==1.1
gast==0.5.3
google-auth==2.3.3
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
gpg==1.15.1
grpcio==1.43.0
h5py==3.6.0
humanize==0.5.1
idna==3.2
ipykernel==6.6.0
ipython==7.30.1
ipython-genutils==0.2.0
ipywidgets==7.6.5
jedi==0.18.1
Jinja2==3.0.3
jsonschema==4.3.2
jupyter==1.0.0
jupyter-client==7.1.0
jupyter-console==6.4.0
jupyter-core==4.9.1
jupyterlab-pygments==0.1.2
jupyterlab-widgets==1.0.2
keras==2.8.0rc0
Keras-Preprocessing==1.1.2
kiwisolver==1.3.2
langtable==0.0.56
libclang==12.0.0
libcomps==0.1.18
lxml==4.6.3
Mako==1.1.4
Markdown==3.3.6
MarkupSafe==2.0.0
matplotlib==3.5.1
matplotlib-inline==0.1.3
mistune==0.8.4
nbclient==0.5.9
nbconvert==6.3.0
nbformat==5.1.3
nest-asyncio==1.5.4
nftables==0.1
notebook==6.4.6
numpy==1.21.5
oauthlib==3.1.1
olefile==0.46
opencv-python==4.5.4.60
opt-einsum==3.3.0
packaging==21.0
pandocfilters==1.5.0
parso==0.8.3
Paste==3.5.0
pexpect==4.8.0
pickleshare==0.7.5
pid==2.2.3
Pillow==8.3.2
pip==21.2.3
ply==3.11
productmd==1.33
prometheus-client==0.12.0
prompt-toolkit==3.0.24
protobuf==3.19.1
ptyprocess==0.6.0
pwquality==1.4.4
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycairo==1.20.1
pycparser==2.20
pycrypto==2.6.1
pycups==2.0.1
pycurl==7.44.1
pyenchant==3.2.2
Pygments==2.10.0
PyGObject==3.42.0
pykickstart==3.34
pyOpenSSL==21.0.0
pyparsing==2.4.7
pyparted==3.11.7
pyrsistent==0.18.0
PySocks==1.7.1
python-augeas==1.1.0
python-dateutil==2.8.1
python-meh==0.50
pyudev==0.22.0
pyxdg==0.27
pyzmq==22.3.0
qtconsole==5.2.2
QtPy==2.0.0
regex==2021.10.23
requests==2.26.0
requests-file==1.5.1
requests-ftp==0.3.1
requests-oauthlib==1.3.0
rpm==4.17.0
rsa==4.8
selinux==3.3
Send2Trash==1.8.0
sepolicy==3.3
setools==4.4.0
setuptools==57.4.0
simpleaudio==1.0.4
simpleline==1.8.2
six==1.16.0
sos==4.2
soupsieve==2.3.1
systemd-python==234
Tempita==0.5.2
tensorboard==2.7.0
tensorboard-data-server==0.6.1
tensorboard-plugin-wit==1.8.0
tensorflow==2.8.0rc0
tensorflow-io-gcs-filesystem==0.23.1
termcolor==1.1.0
terminado==0.12.1
testpath==0.5.0
tf-estimator-nightly==2.8.0.dev2021122109
tornado==6.1
tqdm==4.62.3
traitlets==5.1.1
typing_extensions==4.0.1
urllib3==1.26.6
wcwidth==0.2.5
webencodings==0.5.1
Werkzeug==2.0.2
wheel==0.37.1
widgetsnbextension==3.5.2
wrapt==1.13.3

Issue description

When running TensorBoard with Python 3.10.1, it immediately fails with the following ImportError:

$ tensorboard --logdir=logs/
Traceback (most recent call last):
  File "/home/simon/.local/bin/tensorboard", line 5, in <module>
    from tensorboard.main import run_main
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/main.py", line 27, in <module>
    from tensorboard import default
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/default.py", line 33, in <module>
    from tensorboard.plugins.audio import audio_plugin
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/plugins/audio/audio_plugin.py", line 23, in <module>
    from tensorboard import plugin_util
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/plugin_util.py", line 20, in <module>
    from tensorboard._vendor.bleach.sanitizer import Cleaner
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/bleach/__init__.py", line 5, in <module>
    from tensorboard._vendor.bleach.linkifier import (
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/bleach/linkifier.py", line 4, in <module>
    from tensorboard._vendor import html5lib
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/__init__.py", line 25, in <module>
    from .html5parser import HTMLParser, parse, parseFragment
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/html5parser.py", line 8, in <module>
    from . import _tokenizer
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_tokenizer.py", line 16, in <module>
    from ._trie import Trie
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_trie/__init__.py", line 3, in <module>
    from .py import Trie as PyTrie
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_trie/py.py", line 6, in <module>
    from ._base import Trie as ABCTrie
  File "/home/simon/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_trie/_base.py", line 3, in <module>
    from collections import Mapping
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib64/python3.10/collections/__init__.py)

Root cause seems to be that the html5lib version used by TensorBoard does not yet include html5lib/html5lib-python#403. For a proper fix, html5lib should be updated to 1.1, which includes the fix.

Workaround: Manually changing from collections import [..] to from collections.abc import [..] in ~/.local/lib/python3.10/site-packages/tensorboard/_vendor/html5lib/_trie/_base.py resolves the issue.

[pindinagesh]

@simonkrenger

Have you tried from collections import Mapping to from collections.abc import Mapping ? and moreover please have a look at similar issue which is described here #69381312. Thanks

[simonkrenger]

@pindinagesh Yes, as written above I can confirm that using

from collections.abc import Mapping

works around the issue (which is also basically what was introduced in html5lib/html5lib-python#403). Thanks.

[bileschi]

Thanks for the report @simonkrenger , it looks like we will need to make a PR similar to #2549 to upgrade html5lib

[bileschi]

Hrm - looks like we can't upgrade to html5lib 1.1 right away - 1.1 is unreleased.

https://github.com/html5lib/html5lib-python/blob/1.1/CHANGES.rst

[nfelt]

I dunno why the release notes say UNRELEASED but it was definitely released - there's a 1.1 tag on GitHub and it's also on PyPI (https://pypi.org/project/html5lib/1.1/).

That said, since we vendor html5lib anyway rather than depending on it, we don't need it to be any particular release - we could use an arbitrary GitHub commit. We just need to update the dependencies here: https://cs.opensource.google/tensorflow/tensorboard/+/master:third_party/python.bzl;l=47-56;drc=c4ad956e4225b2b85330002745cff09f9e06b2f6

We'll probably also need to update third_party/html5lib.BUILD to match the expected set of files.

[nfelt]

@bmd3k This should be a pretty straightforward fix - let's try to get this into the coming release as well?

[bmd3k]

Additional note: On master branch it does not have the UNRELEASED text. It just seems like the 1.1 branch does not document its own release :).