Skip to content

Move CI to GitHub Actions & split python versions into jobs #603

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

Conversation

browniebroke
Copy link

@browniebroke browniebroke commented Mar 21, 2022

This pull request builds on top of #601. It is an attempt at rethinking the CI pipeline by splitting each Python version on a separate job.

Given that this project might be supporting EOL Python versions beyond what GitHub is willing to do, I've setup all the jobs to run in Docker, even though only Python 2.6 doesn't seem supported. Sadly, I think this is one of the rare versions to not have an official Docker image.

I've spent some time trying to build one from scratch, but it's much harder than I would expect it to be, mainly because get-pip.py isn't working anymore. Here is what I've tried which didn't work:

  • build a Docker image for 2.6 by changing the version in the official Python images
  • using pyenv

From what I can see, the image jmadler/python-future-builder:latest worked at the time, because it was built before PyPI changed something around their certificates last year.

To workaround this, I'm using that image (which has a lot more than we need), and tell pyenv to use Python 2.6. We cannot do pip install <package name> as it fails with SSL errors, hence I'm downloading the packages with curl (I took the URLs from the latest green build in Travis), and installing from tarballs/wheels.

I think the tests are now running on all versions, but are broken on Python 2.6, 3.8 and 3.9.

I got rid of the linting script as it was also failing, I can add it back as a separate job (or using pre-commit.ci), but it's probably a separate discussion.

@browniebroke
Copy link
Author

To workaround this, I'm using that image (which has a lot more than we need), and tell pyenv to use Python 2.6. We cannot do pip install <package name> as it fails with SSL errors, hence I'm downloading the packages with curl (I took the URLs from the latest green build in Travis), and installing from tarballs/wheels.

I've switched to a lighter image mrupgrade/deadsnakes:2.6 which has Python 2.6 pre-installed.

Comment on lines +30 to +41
- if: ${{ matrix.versions.python != '2.6' }}
run: |
docker build \
. \
--build-arg PYTHON_VERSION=${{ matrix.versions.python }} \
-t jmadler/python-future-builder:${{ matrix.versions.python }}
- if: ${{ matrix.versions.python == '2.6' }}
run: |
docker build \
. \
-f 2.6.Dockerfile \
-t jmadler/python-future-builder:${{ matrix.versions.python }}
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you want to push the images to dockerhub, you might want to use this GitHub action, but it'll require a maintainer to setup the credentials.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM. @edschofield you may also want to retag as under the PythonCharmers org instead of under my personal profile.

@jmadler
Copy link
Contributor

jmadler commented May 15, 2022

This approach LGTM. I'm ok to shave off linting, building for distribution, and fixing remaining tests in error as future work.

Comment on lines +30 to +41
- if: ${{ matrix.versions.python != '2.6' }}
run: |
docker build \
. \
--build-arg PYTHON_VERSION=${{ matrix.versions.python }} \
-t jmadler/python-future-builder:${{ matrix.versions.python }}
- if: ${{ matrix.versions.python == '2.6' }}
run: |
docker build \
. \
-f 2.6.Dockerfile \
-t jmadler/python-future-builder:${{ matrix.versions.python }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM. @edschofield you may also want to retag as under the PythonCharmers org instead of under my personal profile.

@browniebroke browniebroke marked this pull request as draft March 21, 2023 15:36
@browniebroke browniebroke marked this pull request as ready for review March 21, 2023 17:42
Copy link
Author

@browniebroke browniebroke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: I fixed the tests for Python 3.8 and fixed a couple of failures for Python 3.9

Comment on lines -694 to -697
# MIME guessing works in Python 3.8!
guessed_mime = None
if sys.hexversion >= 0x03080000:
guessed_mime = "image/gif"
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's no longer working apparently

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if sys.version_info >= (3, 8):

Copy link
Author

@browniebroke browniebroke Mar 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's a problem with the condition: the guessed_mime is always None, even on Python 3.8 or 3.9.

My "no longer working" was referring to the code comment:

# MIME guessing works in Python 3.8!

Comment on lines +5 to +7
# _dummy_thread and dummy_threading modules
# have both been removed in Python 3.9
from _thread import *
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment on lines +1310 to +1313
# Changed in version 3.8: For int operands, the three-argument form
# of pow now allows the second argument to be negative, permitting
# computation of modular inverses.
self.assertEqual(pow(-1, -2, 3), 1)
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

edschofield added a commit that referenced this pull request Feb 21, 2024
@edschofield
Copy link
Contributor

Merged (somewhat manually). Thanks, Bruno!

@browniebroke browniebroke deleted the gh/matrix-dockerfiles branch February 21, 2024 09:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants