Skip to content

deps: Drop support for Python 3.7 and 3.8 #337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 48 commits into from
May 7, 2025
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
9b32164
feat: Drop support for Python 3.7 and 3.8
google-labs-jules[bot] Apr 17, 2025
dccef48
Updates python version in lint.yml
chalmerlowe Apr 18, 2025
f81c738
Updates owlbot, removing reference to 3.8
chalmerlowe Apr 18, 2025
5cb30ca
Updates CONTRIBUTING.rst
chalmerlowe Apr 18, 2025
7f44424
updates pytest warnings
chalmerlowe Apr 18, 2025
3e4e6f6
Removes test_samples-impl ref to older virtualenv package
chalmerlowe Apr 18, 2025
7beea0d
Removes references to pandas older than 1.5.0
chalmerlowe Apr 18, 2025
32d7b8f
Removes pandas older than 1.5 and misc changes
chalmerlowe Apr 18, 2025
e871d73
updates pandas in setup.py
chalmerlowe Apr 18, 2025
2bdf3ef
more updates related to pandas
chalmerlowe Apr 18, 2025
d902e31
still broken
chalmerlowe Apr 18, 2025
a0543b9
Updates FutureWarning tests to account for unittest coverage
chalmerlowe Apr 23, 2025
6a5d943
Updates json array type tests to account for unittest coverage
chalmerlowe Apr 23, 2025
1bbd5f7
updates python version checks to ensure coverage
chalmerlowe Apr 23, 2025
96a5e97
update json test for unittest coverage
chalmerlowe Apr 23, 2025
8af9bee
Update pandas_backports unittests to ensure coverage
chalmerlowe Apr 24, 2025
78574c0
Updates per review comments
chalmerlowe Apr 24, 2025
9ca34b8
Merge branch 'main' into remove-python-37-38
chalmerlowe Apr 24, 2025
de202f4
moves class from version specific compliance file to generic file
chalmerlowe Apr 30, 2025
fd9dd27
Removes weird cut and paste error
chalmerlowe Apr 30, 2025
bfe9115
fix linting errors
chalmerlowe Apr 30, 2025
91e437e
updates import statement to ensure import of JSONArrowType
chalmerlowe Apr 30, 2025
eca1ab7
Revise required github status checks
chalmerlowe Apr 30, 2025
06b4e88
update linting
chalmerlowe Apr 30, 2025
d481be4
temporarily marking a class as no cover
chalmerlowe May 1, 2025
f4a9fc6
more updates
chalmerlowe May 1, 2025
7585a86
marked several snippets as pragma no cover
chalmerlowe May 1, 2025
c777141
updates linting
chalmerlowe May 1, 2025
7d745b0
Updates constraints and setup.py
chalmerlowe May 1, 2025
9511e7a
migrates class from one time compliance file to another
chalmerlowe May 1, 2025
9638e43
updating pyarrow version
chalmerlowe May 1, 2025
d433fae
Updates linting
chalmerlowe May 1, 2025
4c522b3
removes determine all and module reload tests
chalmerlowe May 7, 2025
cbfcd1a
updates re: ndarrybackedextensionarray
chalmerlowe May 7, 2025
2f9fc09
testing blacken as part of owlbot processing using 3.8
chalmerlowe May 7, 2025
7b967e7
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 7, 2025
18098f0
updates blacken to 3.10
chalmerlowe May 7, 2025
20e6565
update python version in lint.yml
chalmerlowe May 7, 2025
f39b97f
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 7, 2025
c23d435
updates owlbot.py
chalmerlowe May 7, 2025
a52c46d
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 7, 2025
4ebe910
updates owlbot.py
chalmerlowe May 7, 2025
cd4942e
testing lint.yml
chalmerlowe May 7, 2025
32245de
testing linting issue
chalmerlowe May 7, 2025
695c33d
testing linting issue
chalmerlowe May 7, 2025
9ea13d3
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] May 7, 2025
2ee0533
updates lint_setup session
chalmerlowe May 7, 2025
f59ee72
Update noxfile.py
chalmerlowe May 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/sync-repo-settings.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ branchProtectionRules:
- 'cla/google'
- 'docs'
- 'lint'
- 'unit (3.8)'
- 'unit (3.9)'
- 'unit (3.10)'
- 'unit (3.11)'
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.8"
python-version: "3.9"
- name: Install nox
run: |
python -m pip install --upgrade setuptools pip wheel
Expand Down
10 changes: 4 additions & 6 deletions .github/workflows/unittest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,37 +5,35 @@
name: unittest
jobs:
unit:
# TODO(https://github.com/googleapis/gapic-generator-python/issues/2303): use `ubuntu-latest` once this bug is fixed.
# Use ubuntu-22.04 until Python 3.7 is removed from the test matrix
# https://docs.github.com/en/actions/using-github-hosted-runners/using-github-hosted-runners/about-github-hosted-runners#standard-github-hosted-runners-for-public-repositories
runs-on: ubuntu-22.04
# Use `ubuntu-latest` runner.
runs-on: ubuntu-latest
strategy:
matrix:
python: ['3.8', '3.9', '3.10', '3.11', '3.12', '3.13']
python: ['3.9', '3.10', '3.11', '3.12', '3.13']
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: Install nox
run: |
python -m pip install --upgrade setuptools pip wheel
python -m pip install nox
- name: Run unit tests
env:
COVERAGE_FILE: .coverage-${{ matrix.python }}
run: |
nox -s unit-${{ matrix.python }}
- name: Upload coverage results
uses: actions/upload-artifact@v4
with:
name: coverage-artifact-${{ matrix.python }}
path: .coverage-${{ matrix.python }}
include-hidden-files: true

unit-prerelease:

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
runs-on: ubuntu-latest
strategy:
matrix:
Expand Down Expand Up @@ -103,7 +101,7 @@
- name: Setup Python
uses: actions/setup-python@v5
with:
python-version: "3.8"
python-version: "3.9"
- name: Install coverage
run: |
python -m pip install --upgrade setuptools pip wheel
Expand Down
40 changes: 0 additions & 40 deletions .kokoro/samples/python3.7/common.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.7/continuous.cfg

This file was deleted.

11 changes: 0 additions & 11 deletions .kokoro/samples/python3.7/periodic-head.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.7/periodic.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.7/presubmit.cfg

This file was deleted.

40 changes: 0 additions & 40 deletions .kokoro/samples/python3.8/common.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.8/continuous.cfg

This file was deleted.

11 changes: 0 additions & 11 deletions .kokoro/samples/python3.8/periodic-head.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.8/periodic.cfg

This file was deleted.

6 changes: 0 additions & 6 deletions .kokoro/samples/python3.8/presubmit.cfg

This file was deleted.

3 changes: 1 addition & 2 deletions .kokoro/test-samples-impl.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,7 @@ export PYTHONUNBUFFERED=1
env | grep KOKORO

# Install nox
# `virtualenv==20.26.6` is added for Python 3.7 compatibility
python3.9 -m pip install --upgrade --quiet nox virtualenv==20.26.6
python3.9 -m pip install --upgrade --quiet nox virtualenv

# Use secrets acessor service account to get secrets
if [[ -f "${KOKORO_GFILE_DIR}/secrets_viewer_service_account.json" ]]; then
Expand Down
14 changes: 5 additions & 9 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ In order to add a feature:
documentation.

- The feature must work fully on the following CPython versions:
3.7, 3.8, 3.9, 3.10, 3.11, 3.12 and 3.13 on both UNIX and Windows.
3.9, 3.10, 3.11, 3.12 and 3.13 on both UNIX and Windows.

- The feature must not add unnecessary dependencies (where
"unnecessary" is of course subjective, but new dependencies should
Expand Down Expand Up @@ -143,12 +143,12 @@ Running System Tests
$ nox -s system

# Run a single system test
$ nox -s system-3.8 -- -k <name of test>
$ nox -s system-3.9 -- -k <name of test>


.. note::

System tests are only configured to run under Python 3.8.
System tests are only configured to run under Python 3.9.
For expediency, we do not run them in older versions of Python 3.

This alone will not run the tests. You'll need to change some local
Expand Down Expand Up @@ -195,11 +195,11 @@ configure them just like the System Tests.

# Run all tests in a folder
$ cd samples/snippets
$ nox -s py-3.8
$ nox -s py-3.9

# Run a single sample test
$ cd samples/snippets
$ nox -s py-3.8 -- -k <name of test>
$ nox -s py-3.9 -- -k <name of test>

********************************************
Note About ``README`` as it pertains to PyPI
Expand All @@ -221,16 +221,12 @@ Supported Python Versions

We support:

- `Python 3.7`_
- `Python 3.8`_
- `Python 3.9`_
- `Python 3.10`_
- `Python 3.11`_
- `Python 3.12`_
- `Python 3.13`_

.. _Python 3.7: https://docs.python.org/3.7/
.. _Python 3.8: https://docs.python.org/3.8/
.. _Python 3.9: https://docs.python.org/3.9/
.. _Python 3.10: https://docs.python.org/3.10/
.. _Python 3.11: https://docs.python.org/3.11/
Expand Down
4 changes: 2 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,11 @@ dependencies.

Supported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^
Python >= 3.7
Python >= 3.9

Unsupported Python Versions
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Python <= 3.6.
Python <= 3.8.


Mac/Linux
Expand Down
49 changes: 18 additions & 31 deletions db_dtypes/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,15 +21,14 @@
import warnings

import numpy
import packaging.version
import pandas
import pandas.api.extensions
from pandas.errors import OutOfBoundsDatetime
import pyarrow
import pyarrow.compute

from db_dtypes import core
from db_dtypes.version import __version__
from db_dtypes.json import JSONArray, JSONDtype, JSONArrowType # noqa: F401

from . import _versions_helpers

Expand All @@ -47,15 +46,6 @@
_NP_BOX_DTYPE = "datetime64[us]"


# To use JSONArray and JSONDtype, you'll need Pandas 1.5.0 or later. With the removal
# of Python 3.7 compatibility, the minimum Pandas version will be updated to 1.5.0.
if packaging.version.Version(pandas.__version__) >= packaging.version.Version("1.5.0"):
Copy link
Contributor

@Linchin Linchin May 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the comments, it seems removing Python 3.7 could ensure pandas >= 1.5.0, and thus ensure JSONArray and JSONDtype to be available. Do we still need _determine_all()?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

from db_dtypes.json import JSONArray, JSONArrowType, JSONDtype
else:
JSONArray = None
JSONDtype = None


@pandas.api.extensions.register_extension_dtype
class TimeDtype(core.BaseDatetimeDtype):
"""
Expand Down Expand Up @@ -347,6 +337,22 @@ def __sub__(self, other):
return super().__sub__(other)


def _determine_all(json_array_type, json_dtype_type):
"""Determines the list for __all__ based on JSON type availability."""
base_all = [
"__version__",
"DateArray",
"DateDtype",
"TimeArray",
"TimeDtype",
]
# Check if both JSON types are available (truthy)
if json_array_type and json_dtype_type:
return base_all + ["JSONDtype", "JSONArray", "JSONArrowType"]
else:
return base_all


def _check_python_version():
"""Checks the runtime Python version and issues a warning if needed."""
sys_major, sys_minor, sys_micro = _versions_helpers.extract_runtime_version()
Expand All @@ -364,23 +370,4 @@ def _check_python_version():

_check_python_version()


if not JSONArray or not JSONDtype:
__all__ = [
"__version__",
"DateArray",
"DateDtype",
"TimeArray",
"TimeDtype",
]
else:
__all__ = [
"__version__",
"DateArray",
"DateDtype",
"JSONDtype",
"JSONArray",
"JSONArrowType",
"TimeArray",
"TimeDtype",
]
__all__ = _determine_all(JSONArray, JSONDtype)
3 changes: 0 additions & 3 deletions db_dtypes/core.py
Original file line number Diff line number Diff line change
Expand Up @@ -186,9 +186,6 @@ def median(
keepdims: bool = False,
skipna: bool = True,
):
if not hasattr(pandas_backports, "numpy_validate_median"):
raise NotImplementedError("Need pandas 1.3 or later to calculate median.")

pandas_backports.numpy_validate_median(
(),
{"out": out, "overwrite_input": overwrite_input, "keepdims": keepdims},
Expand Down
9 changes: 7 additions & 2 deletions db_dtypes/json.py
Original file line number Diff line number Diff line change
Expand Up @@ -277,5 +277,10 @@ def to_pandas_dtype(self):


# Register the type to be included in RecordBatches, sent over IPC and received in
# another Python process.
pa.register_extension_type(JSONArrowType())
# another Python process. Also handle potential pre-registration
try:
pa.register_extension_type(JSONArrowType())
except pa.ArrowKeyError:
# Type 'dbjson' might already be registered if the module is reloaded,
# which is okay.
pass
Loading