Skip to content

adding python3.10 #324

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Apr 17, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ jobs:
strategy:
max-parallel: 4
matrix:
python-version: [3.7, 3.8, 3.9]
python-version: ['3.7', '3.8', '3.9', '3.10']

steps:
- name: Checkout
Expand All @@ -68,7 +68,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
runtime-param: [3.7, 3.8, 3.9]
runtime-param: ['3.7', '3.8', '3.9', '3.10']
steps:
- name: Checkout
uses: actions/checkout@v3
Expand Down
15 changes: 15 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": "Python: Module",
"type": "python",
"request": "launch",
"module": "unittest",
"justMyCode": true
}
]
}
5 changes: 4 additions & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
{
"python.pythonPath": "/usr/local/bin/python3",
"python.formatting.provider": "black"
"python.formatting.provider": "black",
"python.testing.unittestArgs": ["-v", "-s", "./tests", "-p", "test*.py"],
"python.testing.pytestEnabled": false,
"python.testing.unittestEnabled": true
}
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
[![Slack](https://chat.datadoghq.com/badge.svg?bg=632CA6)](https://chat.datadoghq.com/)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/DataDog/datadog-lambda-python/blob/main/LICENSE)

Datadog Lambda Library for Python (3.7, 3.8, and 3.9) enables [enhanced Lambda metrics](https://docs.datadoghq.com/serverless/enhanced_lambda_metrics), [distributed tracing](https://docs.datadoghq.com/serverless/distributed_tracing), and [custom metric submission](https://docs.datadoghq.com/serverless/custom_metrics) from AWS Lambda functions.
Datadog Lambda Library for Python (3.7, 3.8, 3.9, and 3.10) enables [enhanced Lambda metrics](https://docs.datadoghq.com/serverless/enhanced_lambda_metrics), [distributed tracing](https://docs.datadoghq.com/serverless/distributed_tracing), and [custom metric submission](https://docs.datadoghq.com/serverless/custom_metrics) from AWS Lambda functions.

## Installation

Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ classifiers = [
"Programming Language :: Python :: 3.7",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"Programming Language :: Python :: 3.10",
]

[tool.poetry.dependencies]
Expand Down
12 changes: 6 additions & 6 deletions scripts/add_new_region.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@
# Copy layers from us-east-1 to new region
# args: [new-region]

set -e
set -e

OLD_REGION='us-east-1'

PYTHON_VERSIONS_FOR_AWS_CLI=("python3.7" "python3.8" "python3.9")
LAYER_NAMES=("Datadog-Python37" "Datadog-Python38" "Datadog-Python39")
LAYER_NAMES=("Datadog-Python37" "Datadog-Python38" "Datadog-Python38-ARM" "Datadog-Python39" "Datadog-Python39-ARM" "Datadog-Python310" "Datadog-Python310-ARM")
PYTHON_VERSIONS_FOR_AWS_CLI=("python3.7" "python3.8" "python3.8" "python3.9" "python3.9" "python3.10" "python3.10")
NEW_REGION=$1

publish_layer() {
Expand Down Expand Up @@ -68,11 +68,11 @@ for layer_name in "${LAYER_NAMES[@]}"; do
fi

# run for each version of layer
for i in $(seq 1 $last_layer_version); do
for i in $(seq 1 $last_layer_version); do
layer_path=$layer_name"_"$i.zip
aws_version_key="${PYTHON_VERSIONS_FOR_AWS_CLI[$j]}"

# download layer versions
# download layer versions
URL=$(AWS_REGION=$OLD_REGION aws lambda get-layer-version --layer-name $layer_name --version-number $i --query Content.Location --output text)
curl $URL -o $layer_path

Expand All @@ -82,4 +82,4 @@ for layer_name in "${LAYER_NAMES[@]}"; do
done

j=$(expr $j + 1)
done
done
2 changes: 1 addition & 1 deletion scripts/build_layers.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ set -e

LAYER_DIR=".layers"
LAYER_FILES_PREFIX="datadog_lambda_py"
AVAILABLE_PYTHON_VERSIONS=("3.7" "3.8" "3.9")
AVAILABLE_PYTHON_VERSIONS=("3.7" "3.8" "3.9" "3.10")

# Determine which Python versions to build layers for
if [ -z "$PYTHON_VERSION" ]; then
Expand Down
2 changes: 1 addition & 1 deletion scripts/check_layer_size.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ MAX_LAYER_UNCOMPRESSED_SIZE_KB=$(expr 24 \* 1024)

LAYER_FILES_PREFIX="datadog_lambda_py"
LAYER_DIR=".layers"
VERSIONS=("3.7" "3.8" "3.9")
VERSIONS=("3.7" "3.8" "3.9" "3.10")

for version in "${VERSIONS[@]}"
do
Expand Down
6 changes: 3 additions & 3 deletions scripts/list_layers.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

set -e

LAYER_NAMES=("Datadog-Python37" "Datadog-Python38" "Datadog-Python38-ARM" "Datadog-Python39" "Datadog-Python39-ARM")
LAYER_NAMES=("Datadog-Python37" "Datadog-Python38" "Datadog-Python38-ARM" "Datadog-Python39" "Datadog-Python39-ARM" "Datadog-Python310" "Datadog-Python310-ARM")
AVAILABLE_REGIONS=$(aws ec2 describe-regions | jq -r '.[] | .[] | .RegionName')
LAYERS_MISSING_REGIONS=()

Expand Down Expand Up @@ -63,6 +63,6 @@ done

if [ ${#LAYERS_MISSING_REGIONS[@]} -gt 0 ]; then
echo "WARNING: Following regions missing layers: ${LAYERS_MISSING_REGIONS[@]}"
echo "Please run ./add_new_region.sh <new_region> to add layers to the missing regions"
echo "Please run ./add_new_region.sh <new_region> to add layers to the missing regions"
exit 1
fi
fi
10 changes: 5 additions & 5 deletions scripts/publish_layers.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,10 @@ set -e
# Makes sure any subprocesses will be terminated with this process
trap "pkill -P $$; exit 1;" INT

PYTHON_VERSIONS_FOR_AWS_CLI=("python3.7" "python3.8" "python3.8" "python3.9" "python3.9")
LAYER_PATHS=(".layers/datadog_lambda_py-amd64-3.7.zip" ".layers/datadog_lambda_py-amd64-3.8.zip" ".layers/datadog_lambda_py-arm64-3.8.zip" ".layers/datadog_lambda_py-amd64-3.9.zip" ".layers/datadog_lambda_py-arm64-3.9.zip")
AVAILABLE_LAYERS=("Datadog-Python37" "Datadog-Python38" "Datadog-Python38-ARM" "Datadog-Python39" "Datadog-Python39-ARM")
ARCHS=("amd64" "amd64" "amd64""amd64" "amd64" "arm64")
PYTHON_VERSIONS_FOR_AWS_CLI=("python3.7" "python3.8" "python3.8" "python3.9" "python3.9" "python3.10" "python3.10")
LAYER_PATHS=(".layers/datadog_lambda_py-amd64-3.7.zip" ".layers/datadog_lambda_py-amd64-3.8.zip" ".layers/datadog_lambda_py-arm64-3.8.zip" ".layers/datadog_lambda_py-amd64-3.9.zip" ".layers/datadog_lambda_py-arm64-3.9.zip" ".layers/datadog_lambda_py-amd64-3.10.zip" ".layers/datadog_lambda_py-arm64-3.10.zip")
AVAILABLE_LAYERS=("Datadog-Python37" "Datadog-Python38" "Datadog-Python38-ARM" "Datadog-Python39" "Datadog-Python39-ARM" "Datadog-Python310" "Datadog-Python310-ARM")
ARCHS=("amd64" "amd64" "amd64""amd64" "amd64" "arm64" "amd64" "arm64")
AVAILABLE_REGIONS=$(aws ec2 describe-regions | jq -r '.[] | .[] | .RegionName')

# Check that the layer files exist
Expand Down Expand Up @@ -129,7 +129,7 @@ do

# This shouldn't happen unless someone manually deleted the latest version, say 28
# and then try to republish it again. The published version is actually be 29, because
# Lambda layers are immutable and AWS will skip deleted version and use the next number.
# Lambda layers are immutable and AWS will skip deleted version and use the next number.
if [ $latest_version -gt $VERSION ]; then
echo "ERROR: Published version $latest_version is greater than the desired version $VERSION!"
echo "Exiting"
Expand Down
7 changes: 4 additions & 3 deletions scripts/run_integration_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ set -e
# These values need to be in sync with serverless.yml, where there needs to be a function
# defined for every handler_runtime combination
LAMBDA_HANDLERS=("async-metrics" "sync-metrics")
RUNTIMES=("python37" "python38" "python39")
RUNTIMES=("python37" "python38" "python39" "python310")

LOGS_WAIT_SECONDS=20

Expand All @@ -31,8 +31,9 @@ mismatch_found=false
python37=("python3.7" "3.7" $(xxd -l 4 -c 4 -p < /dev/random))
python38=("python3.8" "3.8" $(xxd -l 4 -c 4 -p < /dev/random))
python39=("python3.9" "3.9" $(xxd -l 4 -c 4 -p < /dev/random))
python310=("python3.10" "3.10" $(xxd -l 4 -c 4 -p < /dev/random))

PARAMETERS_SETS=("python37" "python38" "python39")
PARAMETERS_SETS=("python37" "python38" "python39" "python310")

if [ -z "$RUNTIME_PARAM" ]; then
echo "Python version not specified, running for all python versions."
Expand Down Expand Up @@ -111,7 +112,7 @@ python version : ${!python_version} and run id : ${!run_id}"
for input_event_file in "${input_event_files[@]}"; do
# Get event name without trailing ".json" so we can build the snapshot file name
input_event_name=$(echo "$input_event_file" | sed "s/.json//")
snapshot_path="./snapshots/return_values/${handler_name}_${parameters_set}_${input_event_name}.json"
snapshot_path="./snapshots/return_values/${handler_name}_${input_event_name}.json"

return_value=$(PYTHON_VERSION=${!python_version} RUNTIME=$parameters_set SERVERLESS_RUNTIME=${!serverless_runtime} \
serverless invoke --stage ${!run_id} -f "$function_name" --path "./input_events/$input_event_file")
Expand Down
4 changes: 2 additions & 2 deletions scripts/run_tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
# Run unit tests in Docker
set -e

PYTHON_VERSIONS=("3.7" "3.8" "3.9")
PYTHON_VERSIONS=("3.7" "3.8" "3.9" "3.10")

for python_version in "${PYTHON_VERSIONS[@]}"
do
Expand All @@ -18,7 +18,7 @@ do
--build-arg python_version=$python_version
docker run -w /test \
datadog-lambda-python-test:$python_version \
nose2 -v
nose2 -v
docker run -w /test \
datadog-lambda-python-test:$python_version \
flake8 datadog_lambda/
Expand Down
6 changes: 4 additions & 2 deletions scripts/sign_layers.sh
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ LAYER_FILES=(
"datadog_lambda_py-arm64-3.8.zip"
"datadog_lambda_py-amd64-3.9.zip"
"datadog_lambda_py-arm64-3.9.zip"
"datadog_lambda_py-amd64-3.10.zip"
"datadog_lambda_py-arm64-3.10.zip"
)
SIGNING_PROFILE_NAME="DatadogLambdaSigningProfile"

Expand All @@ -38,7 +40,7 @@ fi

for LAYER_FILE in "${LAYER_FILES[@]}"
do
echo
echo
echo "${LAYER_FILE}"
echo "-------------------------"

Expand Down Expand Up @@ -68,7 +70,7 @@ do
do
sleep 3
SECONDS_WAITED_SO_FAR=$((SECONDS_WAITED_SO_FAR + 3))

SIGNING_JOB_DESCRIPTION=$(aws signer describe-signing-job \
--job-id $SIGNING_JOB_ID \
--region $REGION\
Expand Down
Loading