`python:3.11.5-slim` seems to break poetry installation

[flxdot]

I noticed today that our CI if failing to install poetry. I could isolate the issue being related to the python:3.11.5-slim image:

Working Example: 3114.Dockerfile

FROM python:3.11.4-slim

RUN apt-get upgrade && apt-get update && apt-get install -y curl python3-dev libpq-dev gcc
# Install Poetry
ENV POETRY_NO_INTERACTION=1 \
    POETRY_VIRTUALENVS_IN_PROJECT=1 \
    POETRY_VIRTUALENVS_CREATE=1 \
    POETRY_VERSION=1.6.1
RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python -

docker build . -f 3114.Dockerfile

[+] Building 15.7s (7/7) FINISHED                                                                                                                                                                                                                  
 => [internal] load build definition from 3114.Dockerfile                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 423B                                                                                                                                                                                                          0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                             0.0s
 => => transferring context: 72B                                                                                                                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/python:3.11.4-slim                                                                                                                                                                         1.3s
 => [1/3] FROM docker.io/library/python:3.11.4-slim@sha256:17d62d681d9ecef20aae6c6605e9cf83b0ba3dc247013e2f43e1b5a045ad4901                                                                                                                   0.0s
 => CACHED [2/3] RUN apt-get upgrade && apt-get update && apt-get install -y curl python3-dev libpq-dev gcc                                                                                                                                   0.0s
 => [3/3] RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python -                                                                                                                                                 13.8s
 => exporting to image                                                                                                                                                                                                                        0.5s
 => => exporting layers                                                                                                                                                                                                                       0.5s
 => => writing image sha256:059ed17a0f156ad39fb26611741d6c464399b6b4191ac8e1d6e66accee639041              

Breaking Example: 3115.Dockerfile

FROM python:3.11.5-slim

RUN apt-get upgrade && apt-get update && apt-get install -y curl python3-dev libpq-dev gcc
# Install Poetry
ENV POETRY_NO_INTERACTION=1 \
    POETRY_VIRTUALENVS_IN_PROJECT=1 \
    POETRY_VIRTUALENVS_CREATE=1 \
    POETRY_VERSION=1.6.1
RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python -

docker build . -f 3115.Dockerfile

[+] Building 19.8s (6/6) FINISHED                                                                                                                                                                                                                  
 => [internal] load build definition from 3115.Dockerfile                                                                                                                                                                                     0.0s
 => => transferring dockerfile: 423B                                                                                                                                                                                                          0.0s
 => [internal] load .dockerignore                                                                                                                                                                                                             0.0s
 => => transferring context: 72B                                                                                                                                                                                                              0.0s
 => [internal] load metadata for docker.io/library/python:3.11.5-slim                                                                                                                                                                         0.7s
 => [1/3] FROM docker.io/library/python:3.11.5-slim@sha256:c4992301d47a4f1d3e73c034494c080132f9a4090703babfcfa3317f7ba54461                                                                                                                   0.0s
 => CACHED [2/3] RUN apt-get upgrade && apt-get update && apt-get install -y curl python3-dev libpq-dev gcc                                                                                                                                   0.0s
 => ERROR [3/3] RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python -                                                                                                                                           19.0s
------                                                                                                                                                                                                                                             
 > [3/3] RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python -:                                                                                                                                                       
#0 18.95 Retrieving Poetry metadata                                                                                                                                                                                                                
#0 18.95                                                                                                                                                                                                                                           
#0 18.95 # Welcome to Poetry!                                                                                                                                                                                                                      
#0 18.95                                                                                                                                                                                                                                           
#0 18.95 This will download and install the latest version of Poetry,
#0 18.95 a dependency and package manager for Python.
#0 18.95 
#0 18.95 It will add the `poetry` command to Poetry's bin directory, located at:
#0 18.95 
#0 18.95 /opt/poetry/bin
#0 18.95 
#0 18.95 You can uninstall at any time by executing this script with the --uninstall option,
#0 18.95 and these changes will be reverted.
#0 18.95 
#0 18.95 Installing Poetry (1.6.1)
#0 18.95 Installing Poetry (1.6.1): Creating environment
#0 18.95 Installing Poetry (1.6.1): Installing Poetry
#0 18.95 Installing Poetry (1.6.1): An error occurred. Removing partial environment.
#0 18.95 Poetry installation failed.
#0 18.95 See /poetry-installer-error-4uad51ii.log for error logs.
------
ERROR: failed to solve: executor failed running [/bin/sh -c curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python -]: exit code: 1

Error Log:

WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
Could not fetch URL https://pypi.org/simple/poetry/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/poetry/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
ERROR: Could not find a version that satisfies the requirement poetry==1.6.1 (from versions: none)
ERROR: No matching distribution found for poetry==1.6.1
WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping

Traceback:

  File "<stdin>", line 923, in main
  File "<stdin>", line 560, in run
  File "<stdin>", line 582, in install
  File "<stdin>", line 685, in install_poetry
  File "<stdin>", line 375, in pip
  File "<stdin>", line 372, in python
  File "<stdin>", line 365, in run

My Environment

Operating System: MacOS Ventura 13.4.1 (c) (22F770820d)
CPU: Apple M1 Pro

Docker: 4.17.0 (99724)

Temporary solution

Pin image version to python:3.11.4-slim

[jaimecp89]

We had the same problem in our CI. However, the issue only appears in the arm64 build (Apple silicon). Everything works fine for amd64: DOCKER_DEFAULT_PLATFORM=linux/amd64 docker build ...

Another temporary solution we found is to replace the python3-dev library in:
RUN apt-get upgrade && apt-get update && apt-get install -y curl python3-dev libpq-dev gcc
with libc6-dev.

I hope this gives a hint on where the problem is 👍

[cuttlefish]

I'm also having a problem (building a different package) because of a missing libc6-dev. I'm not sure how the build works for these images, but it certainly looks like libc6-dev should be getting installed. Are these images actually generated from this repo?

python/3.11/slim-bookworm/Dockerfile

Line 39 in 81b6e5f

libc6-dev \

For me, it was failing with amd64 as well. The image that I pulled doesn't seem to have the libc6-dev package either. Although I may have bungled pulling the correct image, I'll try it on a proper amd64 machine later.

3.11.4-slim-bookworm had libc6-dev installed.

[Tsutomu-Ikeda]

I have the same problem with python:3.11.6-slim

python -c 'import ssl; print(ssl.OPENSSL_VERSION)': OpenSSL 3.0.11 19 Sep 2023
python --version: Python 3.11.6

Error log:

18.06 WARNING: pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
18.06 WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
18.06 WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
18.06 WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
18.06 WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
18.06 WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.")': /simple/poetry/
18.06 Could not fetch URL https://pypi.org/simple/poetry/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/poetry/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.")) - skipping
18.06 ERROR: Could not find a version that satisfies the requirement poetry==1.5.1 (from versions: none)
18.06 ERROR: No matching distribution found for poetry==1.5.1
18.06 
18.06 Traceback:
18.06 
18.06   File "<stdin>", line 923, in main
18.06   File "<stdin>", line 560, in run
18.06   File "<stdin>", line 582, in install
18.06   File "<stdin>", line 685, in install_poetry
18.06   File "<stdin>", line 375, in pip
18.06   File "<stdin>", line 372, in python
18.06   File "<stdin>", line 365, in run

[LaurentGoderre]

Earlier versions of 3.11 accidentally included dev packages that weren't mean to be part of the image and it was fix in #858. I recommend you add libc6-dev to your image if one of your dependency requires it.

[tianon]

See also #882, especially #882 (comment)