Botocore raises an issue about quote_from_bytes() expected bytes

[lehmat]

This issue was first raised today and is working for a release done 6 days ago (6th of Sep). Some dependency(?) has been upgraded between 6 days ago and now and it does not work well with pynamodb.

TypeError: quote_from_bytes() expected bytes
  File "botocore/httpsession.py", line 409, in send
    urllib_response = conn.urlopen(
  File "urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "urllib3/connection.py", line 239, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "botocore/awsrequest.py", line 92, in _send_request
    rval = super(AWSConnection, self)._send_request(
  File "http/client.py", line 1267, in _send_request
    self.putrequest(method, url, **skips)
  File "urllib3/connection.py", line 219, in putrequest
    return _HTTPConnection.putrequest(self, method, url, *args, **kwargs)
  File "urllib/parse.py", line 861, in quote
    return quote_from_bytes(string, safe)
  File "urllib/parse.py", line 886, in quote_from_bytes
    raise TypeError("quote_from_bytes() expected bytes")
HTTPClientError: An HTTP Client raised an unhandled exception: quote_from_bytes() expected bytes
  File "pynamodb/connection/base.py", line 548, in get_meta_table
    data = self.dispatch(DESCRIBE_TABLE, operation_kwargs)
  File "pynamodb/connection/base.py", line 329, in dispatch
    data = self._make_api_call(operation_name, operation_kwargs, settings)
  File "pynamodb/connection/base.py", line 386, in _make_api_call
    http_response = self.client._endpoint.http_session.send(prepared_request)
  File "aws_xray_sdk/ext/pynamodb/patch.py", line 50, in _xray_traced_pynamodb
    return xray_recorder.record_subsegment(
  File "aws_xray_sdk/core/recorder.py", line 436, in record_subsegment
    return_value = wrapped(*args, **kwargs)
  File "botocore/httpsession.py", line 454, in send
    raise HTTPClientError(error=e)
TableError: Unable to describe table: An HTTP Client raised an unhandled exception: quote_from_bytes() expected bytes
  File "flask/app.py", line 2525, in wsgi_app
    response = self.full_dispatch_request()
  File "flask/app.py", line 1822, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "flask/app.py", line 1820, in full_dispatch_request
    rv = self.dispatch_request()
  File "flask/app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "views.py", line 22, in get_data
    data= MyDynamoModel.get(
  File "pynamodb/models.py", line 540, in get
    data = cls._get_connection().get_item(
  File "pynamodb/connection/table.py", line 213, in get_item
    return self.connection.get_item(
  File "pynamodb/connection/base.py", line 1197, in get_item
    operation_kwargs = self.get_operation_kwargs(
  File "pynamodb/connection/base.py", line 903, in get_operation_kwargs
    operation_kwargs.update(self.get_identifier_map(table_name, hash_key, range_key, key=key))
  File "pynamodb/connection/base.py", line 820, in get_identifier_map
    tbl = self.get_meta_table(table_name)
  File "pynamodb/connection/base.py", line 551, in get_meta_table
    raise TableError("Unable to describe table: {}".format(e), e)

[lehmat]

boto3==1.24.61 <- This was during last build time the latest
boto3==1.24.73 <- This failed (Automatically coming from lambda layer)

[lehmat]

confirmed that pinning the following requirements work in new release:

boto3==1.24.66
botocore==1.27.66

[lehmat]

Finally, was able to fix this. This was NOT anyhow tied to botocore or any other place related to pynamodb. This just was arised from this - which made me think it was it.

I had to disable AWS X-ray to get the proper root cause to popup. Link to the ticket sentry ticket (1626)