Automated PyPi Publish

[kchason]

• Builds and uploads wheels to GitHub on all pull requests and commits to develop or main
• Uploads wheel to PyPi on tag

Note, a GitHub Actions secret is required to be created called PYPI_API_TOKEN

[ajnelson-nist]

I think it is not a good idea to post to PyPI on every push to develop, or even pushes to main. We've recently encountered reasons to push to main but not issue a release, e.g. for updating Github templates.

Please rescope this action to only push on generating a release. Releases should be documented, and I'd like to know what the tie is between Github's release description box and an automated PyPI upload.

You added these steps into ci.yml. Would it be appropriate to instead create a file cd.yml, to emphasize the Delivery part of CI/CD?

I also see a "legacy" term I don't quite understand in the twine command. Is there documentation you followed to generate these GIthub Actions?

[kchason]

So it only pushes to PyPi when tagged, but it builds the wheels and maintains them in the workflow for manual download if desired, though I can remove that part if it doesn't make sense. The description is defined in setup.cfg to be the README.md file and that's what PyPi will display as well.

We can split the pipeline into CI vs CD, but it makes the triggering a bit more complicated if we (and I assume we do) want the tests to run before packaging and uploading to PyPi. We could alternatively update the name of the file to something more generic such as workflow.yml to be more representative of the scope.

There is, and I have to find it, but the short of it was that the GitHub Action for uploading to PyPi didn't work the way I expected it to when setting up the automated process for SQLite Dissect and had followed conventions for utilizing twine for it.

[ajnelson-nist]

Re:

it only pushes to PyPi when tagged,

Can you explain why the last patch removed a tag constraint?

[kchason]

Can you explain why the last patch removed a tag constraint?

It removed it from the build, but not from the push. It only gets pushed to PyPi on tags, but still builds the wheels locally on every pipeline execution.

if: startsWith(github.ref, 'refs/tags')

[ajnelson-nist]

Can you explain why the last patch removed a tag constraint?

It removed it from the build, but not from the push. It only gets pushed to PyPi on tags, but still builds the wheels locally on every pipeline execution.

if: startsWith(github.ref, 'refs/tags')

Thank you, I'm now good on this concern.

Back to prior replies:

So it only pushes to PyPi when tagged, but it builds the wheels and maintains them in the workflow for manual download if desired, though I can remove that part if it doesn't make sense. ...

I was a bit confused on the upload-artifact action earlier. It looks like this upload on every CI passing is done to Github, not to PyPI. I'm still not convinced that's helpful, because that URL seems difficult to predict, but for now I don't see the harm in it either, so we can start out with it.

... The description is defined in setup.cfg to be the README.md file and that's what PyPi will display as well.

Oh right, I left that as an ancient TODO, because I'm not sure how it'll display. I'll try using twine and the Test PyPI service once before we let this upload, so we can avoid the 5-minutes later x.y.1 release.

We can split the pipeline into CI vs CD, but it makes the triggering a bit more complicated if we (and I assume we do) want the tests to run before packaging and uploading to PyPi. We could alternatively update the name of the file to something more generic such as workflow.yml to be more representative of the scope.

I personally prefer cicd.yml to maintain some scope of purpose. But splitting sounds unnecessarily tricky, I didn't appreciate that before.

There is, and I have to find it, but the short of it was that the GitHub Action for uploading to PyPi didn't work the way I expected it to when setting up the automated process for SQLite Dissect and had followed conventions for utilizing twine for it.

I saw the legacy term used in that Twine documentation you cited, so I won't note that further.

Summary: Let's settle on the name of the YAML file, and then I'm fine with this merging. I might have another PR to revise the documentation render after trying the test PyPI service.