Latest version of requirements not being installed in CI

[emilykl]

Recent CI runs are installing old versions of many packages, despite no version pins:

Click to expand

Resolved 3 packages in 24ms
      Built plotly @ file:///home/circleci/project
Prepared 3 packages in 28.10s
Installed 3 packages in 351ms
 + narwhals==1.29.1
 + packaging==24.2
 + plotly==6.0.0 (from file:///home/circleci/project)
Resolved 229 packages in 2.23s
      Built curio==1.6
      Built aplus==0.11.0
Prepared 227 packages in 6.54s
Installed 228 packages in 273ms
 + alabaster==0.7.13
 + annotated-types==0.7.0
 + anyio==4.5.2
 + anywidget==0.9.15
 + aplus==0.11.0
 + argon2-cffi==23.1.0
 + argon2-cffi-bindings==21.2.0
 + arrow==1.3.0
 + astropy==5.2.2
 + asttokens==3.0.0
 + async-lru==2.0.4
 + attrs==25.1.0
 + babel==2.17.0
 + backcall==0.2.0
 + backports-zoneinfo==0.2.1
 + beautifulsoup4==4.13.3
 + black==24.8.0
 + blake3==1.0.4
 + bleach==6.1.0
 + bqplot==0.12.44
 + branca==0.8.1
 + cachetools==5.5.2
 + certifi==2025.1.31
 + cffi==1.17.1
 + charset-normalizer==3.4.1
 + click==8.1.8
 + click-plugins==1.1.1
 + cligj==0.7.2
 + cloudpickle==3.1.1
 + comm==0.2.2
 + contourpy==1.1.1
 + curio==1.6
 + cycler==0.12.1
 + dask==2023.5.0
 + debugpy==1.8.13
 + decorator==5.2.1
 + defusedxml==0.7.1
 + docrepr==0.2.0
 + docutils==0.20.1
 + exceptiongroup==1.2.2
 + executing==2.2.0
 + fastapi==0.115.11
 + fastjsonschema==2.21.1
 + filelock==3.16.1
 + fiona==1.10.1
 + fonttools==4.56.0
 + fqdn==1.5.1
 + frozendict==2.4.6
 + fsspec==2025.2.0
 + future==1.0.0
 + geopandas==0.13.2
 + h11==0.14.0
 + h5py==3.11.0
 + httpcore==1.0.7
 + httptools==0.6.4
 + httpx==0.28.1
 + idna==3.10
 + imageio==2.35.1
 + imagesize==1.4.1
 + importlib-metadata==8.5.0
 + importlib-resources==6.4.5
 + iniconfig==2.0.0
 + ipydatawidgets==4.3.5
 + ipykernel==6.29.5
 + ipyleaflet==0.19.2
 + ipympl==0.9.3
 + ipyparallel==9.0.1
 + ipython==8.12.3
 + ipython-genutils==0.2.0
 + ipyvolume==0.6.3
 + ipyvue==1.11.2
 + ipyvuetify==1.11.1
 + ipywebrtc==0.6.0
 + ipywidgets==8.1.5
 + isoduration==20.11.0
 + jedi==0.19.2
 + jinja2==3.1.6
 + json5==0.10.0
 + jsonpointer==3.0.0
 + jsonschema==4.23.0
 + jsonschema-specifications==2023.12.1
 + jupyter==1.1.1
 + jupyter-client==8.6.3
 + jupyter-console==6.6.3
 + jupyter-core==5.7.2
 + jupyter-events==0.10.0
 + jupyter-leaflet==0.19.2
 + jupyter-lsp==2.2.5
 + jupyter-server==2.14.2
 + jupyter-server-terminals==0.5.3
 + jupyterlab==4.3.5
 + jupyterlab-pygments==0.3.0
 + jupyterlab-server==2.27.3
 + jupyterlab-widgets==3.0.13
 + kaleido==0.2.1
 + kiwisolver==1.4.7
 + lazy-loader==0.4
 + llvmlite==0.41.1
 + locket==1.0.0
 + markdown-it-py==3.0.0
 + markupsafe==2.1.5
 + matplotlib==3.7.5
 + matplotlib-inline==0.1.7
 + mdurl==0.1.2
 + mistune==3.1.2
 + mypy-extensions==1.0.0
 + nbclient==0.10.1
 + nbconvert==7.16.6
 + nbformat==5.10.4
 + nest-asyncio==1.6.0
 + networkx==3.1
 + notebook==7.3.2
 + notebook-shim==0.2.4
 + numba==0.58.1
 + numpy==1.24.4
 + orjson==3.10.15
 + outcome==1.3.0.post0
 + overrides==7.7.0
 + pandas==2.0.3
 + pandocfilters==1.5.1
 + parso==0.8.4
 + partd==1.4.1
 + pathspec==0.12.1
 + patsy==1.0.1
 + pexpect==4.9.0
 + pickleshare==0.7.5
 + pillow==10.4.0
 + pkgutil-resolve-name==1.3.10
 + platformdirs==4.3.6
 + plotly-geo==1.0.0
 + pluggy==1.5.0
 + polars==1.8.2
 + progressbar2==4.5.0
 + prometheus-client==0.21.1
 + prompt-toolkit==3.0.50
 + psutil==7.0.0
 + psygnal==0.11.1
 + ptyprocess==0.7.0
 + pure-eval==0.2.3
 + py==1.11.0
 + pyarrow==17.0.0
 + pycparser==2.22
 + pydantic==2.10.6
 + pydantic-core==2.27.2
 + pyerfa==2.0.0.3
 + pygments==2.19.1
 + pyparsing==3.1.4
 + pyproj==3.5.0
 + pyshp==2.3.1
 + pytest==6.2.5
 + pytest-asyncio==0.20.3
 + python-dateutil==2.9.0.post0
 + python-dotenv==1.0.1
 + python-json-logger==3.3.0
 + python-utils==3.8.2
 + pythreejs==2.4.2
 + pytz==2025.1
 + pywavelets==1.4.1
 + pyyaml==6.0.2
 + pyzmq==26.2.1
 + qtconsole==5.6.1
 + qtpy==2.4.3
 + referencing==0.35.1
 + requests==2.32.3
 + rfc3339-validator==0.1.4
 + rfc3986-validator==0.1.1
 + rich==13.9.4
 + rpds-py==0.20.1
 + scikit-image==0.21.0
 + scipy==1.10.1
 + send2trash==1.8.3
 + setuptools==75.3.0
 + shapely==2.0.7
 + six==1.17.0
 + sniffio==1.3.1
 + snowballstemmer==2.2.0
 + sortedcontainers==2.4.0
 + soupsieve==2.6
 + sphinx==7.1.2
 + sphinx-rtd-theme==3.0.2
 + sphinxcontrib-applehelp==1.0.4
 + sphinxcontrib-devhelp==1.0.2
 + sphinxcontrib-htmlhelp==2.0.1
 + sphinxcontrib-jquery==4.1
 + sphinxcontrib-jsmath==1.0.1
 + sphinxcontrib-qthelp==1.0.3
 + sphinxcontrib-serializinghtml==1.1.5
 + stack-data==0.6.3
 + starlette==0.44.0
 + statsmodels==0.14.1
 + tabulate==0.9.0
 + terminado==0.18.1
 + testpath==0.6.0
 + tifffile==2023.7.10
 + tinycss2==1.2.1
 + toml==0.10.2
 + tomli==2.2.1
 + toolz==1.0.0
 + tornado==6.4.2
 + tqdm==4.67.1
 + traitlets==5.14.3
 + traittypes==0.2.1
 + trio==0.27.0
 + types-python-dateutil==2.9.0.20241206
 + typing-extensions==4.12.2
 + tzdata==2025.1
 + uri-template==1.3.0
 + urllib3==2.2.3
 + uvicorn==0.33.0
 + uvloop==0.21.0
 + vaex==4.17.0
 + vaex-astro==0.9.3
 + vaex-core==4.17.1
 + vaex-hdf5==0.14.1
 + vaex-jupyter==0.8.2
 + vaex-ml==0.18.3
 + vaex-server==0.9.0
 + vaex-viz==0.5.4
 + watchfiles==0.24.0
 + wcwidth==0.2.13
 + webcolors==24.8.0
 + webencodings==0.5.1
 + websocket-client==1.8.0
 + websockets==13.1
 + widgetsnbextension==4.0.13
 + xarray==2023.1.0
 + xyzservices==2025.1.0
 + zipp==3.20.2
npm warn deprecated mumath@3.3.4: Redundant dependency in your project.
npm warn deprecated @types/sass@1.45.0: This is a stub types definition. sass provides its own type definitions, so you do not need this installed.

added 424 packages, and audited 425 packages in 7s

47 packages are looking for funding
  run `npm fund` for details

2 moderate severity vulnerabilities

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.

> build
> esbuild --bundle --alias:plotly.js=plotly.js/dist/plotly --format=esm --minify --outfile=../plotly/package_data/widgetbundle.js widget.ts


  ../plotly/package_data/widgetbundle.js  4.6mb ⚠️

⚡ Done in 999ms

pytest for example is getting installed as version 6.2.5 even though the latest is 8.3.5.

I'm not quite sure why this is happening, but I'm able to recreate the behavior on my local machine, with both pip and uv.

At first I thought it was due to a dependency conflict, but pinning pytest==8.3.5 results in a successful resolution, and the correct (newer) pytest version is installed. So there is no package forcing an older pytest. Using the --no-cache flag with uv doesn't seem to change the behavior either.

I suppose we could just set lower bounds on some of the dependencies. But I'd like to understand why this is happening.

[emilykl]

It looks like something is causing the resolver to choose ipython==8.12.3, which requires pytest<7.

Why? I have no idea, but changing ipython[all] to ipython in the requirements file results in the latest pytest being installed (although ipython==8.12.3 is still installed despite not being the latest)... so apparently one of the extra dependencies specified by all is leading the resolver down a bad path...

Possibly the pytest-asyncio<0.22 pin? Since it's the only extra dependency with an upper bound on that list.

[emilykl]

one clue -- ipython==8.13.2 is the last version of ipython which supports Python 3.8, and I'm testing in a Python 3.8 env so that does explain why older ipython versions are being used...