Docs: Packages installed in the layer are different that the ones from `aws_lambda_powertools[all]`

[heitorlessa]

Summary: Converting discussion into a doc issue so we can prioritize it in this iteration. We answered the discussion on why that is.

Discussed in #1807

^{Originally posted by ericbn January 4, 2023}
Hi! 👋

Below is the list of installed packaged I get with a lambda using the arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV2:17 layer, obtained with this code:

import pkg_resources


def lambda_handler(event, context):
    print(*sorted(f"{i.key}=={i.version}" for i in pkg_resources.working_set), sep="\n")

Output:

aws-lambda-powertools==2.5.0
aws-xray-sdk==2.11.0
awslambdaric==2.0.4
boto3==1.20.32
botocore==1.23.32
fastjsonschema==2.16.2
jmespath==0.10.0
pip==22.0.4
pydantic==1.10.2
python-dateutil==2.8.2
s3transfer==0.5.2
setuptools==58.1.0
simplejson==3.17.2
six==1.16.0
typing-extensions==4.4.0
urllib3==1.26.9
wrapt==1.14.1

And these are the packages I get by installing aws_lambda_powertools[all,aws-sdk]==2.5.0:

❯ python3.9 -m venv venv
❯ . venv/bin/activate
❯ pip install 'aws_lambda_powertools[all,aws-sdk]==2.5.0'
❯ pip freeze --local
aws-lambda-powertools==2.5.0
aws-xray-sdk==2.11.0
boto3==1.26.42
botocore==1.29.42
fastjsonschema==2.16.2
jmespath==1.0.1
pydantic==1.10.4
python-dateutil==2.8.2
s3transfer==0.6.0
six==1.16.0
typing_extensions==4.4.0
urllib3==1.26.13
wrapt==1.14.1

Here's the difference (removed pip and setuptools from the left, which is the list from the layer):

Questions:

1. Shouldn't the package versions all be the same between the layer and the aws_lambda_powertools extra(s)?
2. Shouldn't awslambdaric and boto3 already be included in aws_lambda_powertools[all] (the "all" extra), or some other extra that is intended to match the packages in the layer?

I'm assuming it's a good practice to add aws_lambda_powertools[all]==2.5.0 to the dev dependencies and then deploy your lambdas only with extra dependencies beyond those already included as dev, because I was assuming those dev dependencies would be the exact same ones already provided in the arn:aws:lambda:us-east-1:017000801446:layer:AWSLambdaPowertoolsPythonV2:17 layer. In other words, I was expecting the diff above to perfectly match. I see it's not the case.

(Side question: Any plans on updating the boto3 version in next releases of the layer and the package?)

[heitorlessa]

Now merged. Attaching screenshots in case anyone else drops by before it's released.

[github-actions]

This is now released under 2.10.0 version!