Bug: JMESPath semantic version mismatch with boto3

[corey-cole]

Expected Behaviour

I should be able to install an unpinned version of boto3 and aws-lambda-powertools without a dependency error

Current Behaviour

Installing boto3 and aws-lambda-powertools==1.25.7 results in the following error message during installation.

aws-lambda-powertools 1.25.7 requires jmespath<0.11.0,>=0.10.0, but you'll have jmespath 1.0.0 which is incompatible.

Code snippet

N/A

Possible Solution

boto3 added support for JMESPath 1.0 in this PR: boto/boto3#3189. Didn't appear to require any code changes, just opening up the allowable versions to something less than 2.0

Steps to Reproduce

cat requirements.txt
wheel
boto3
aws-lambda-powertools[pydantic]==1.25.7

pip install -r requirements.txt

AWS Lambda Powertools for Python version

1.25.7

AWS Lambda function runtime

3.9

Packaging format used

PyPi

Debugging logs

No response

[boring-cyborg]

Thanks for opening your first issue here! We'll come back to you as soon as we can.

[michaelbrewer]

@corey-cole works for me using Python 3.9.12 and using the same requirements file

[michaelbrewer]

Maybe you mean when you specially want to use jmespath==1.0.0 along with aws-lambda-powertools[pydantic]==1.25.7

requirements.txt:

boto3
jmespath==1.0.0
aws-lambda-powertools[pydantic]==1.25.7

Then you would be able to reproduce this:

(.venv) ➜  xx pip install -r requirements.txt
Collecting wheel
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Collecting boto3
  Using cached boto3-1.21.44-py3-none-any.whl (132 kB)
Collecting jmespath==1.0.0
  Using cached jmespath-1.0.0-py3-none-any.whl (23 kB)
Collecting aws-lambda-powertools[pydantic]==1.25.7
  Using cached aws_lambda_powertools-1.25.7-py3-none-any.whl (171 kB)
Collecting fastjsonschema<3.0.0,>=2.14.5
  Using cached fastjsonschema-2.15.3-py3-none-any.whl (22 kB)
Collecting aws-xray-sdk<3.0.0,>=2.8.0
  Using cached aws_xray_sdk-2.9.0-py2.py3-none-any.whl (98 kB)
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of jmespath to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install aws-lambda-powertools[pydantic]==1.25.7 and jmespath==1.0.0 because these package versions have conflicting dependencies.

The conflict is caused by:
    The user requested jmespath==1.0.0
    aws-lambda-powertools[pydantic] 1.25.7 depends on jmespath<0.11.0 and >=0.10.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

[michaelbrewer]

Finally it looks like jmespath.py has dropped support for Python 3.6:

• jmespath/jmespath.py@0.10.0...1.0.0

[michaelbrewer]

jmespath = [
  { version ="<=0.10.0", python = "^3.6" },
  { version ="^1.0.0", python = "^3.7" }
]

@corey-cole a possible solution can be made ie: #1139

[heitorlessa]

Thanks for reporting this issue @corey-cole and thanks for sending a quick dep range fix @michaelbrewer.

We were conservative in adding jmespath as an explicit dependency after exposing custom JMESPath functions instead of relying on boto3 vending jmespath.

I took a look at boto and they initially added jmespath 1.0 only, then corrected their markers in a subsequent release as it broke to <3.6 etc. Looking at jmespath 1.0 it's only dropping 3.6.

Cutting a release ASAP.

Thank you!

[github-actions]

This is now released under 1.25.8 version!

[heitorlessa]

While it worked locally before release, it does sound like the markers aren't being honored.

requirements.txt

boto3
aws-lambda-powertools==1.25.8

Py 3.9 output

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
aws-lambda-powertools 1.25.8 requires jmespath<=0.10.0; python_version >= "3.6" and python_version < "4.0", but you have jmespath 1.0.0 which is incompatible.