url parsing struggle with username containing @ char

[victornoel]

• asyncpg version: 0.18.3

I am using asyncpg to connect to a Azure PostgreSQL database, and Azure imposes the use of a username of the form name@db, so the full connection url looks like:

postgresql://my-user@my-db-server:my-password@my-db-server.postgres.database.azure.com:5432/some-db

When using such an url to connect with asyncpg, I get the following error:

File "/home/victor/.local/share/virtualenvs/datapipe-3ZEOKy8g/lib/python3.7/site-packages/asyncpg/connect_utils.py", line 187, in _parse_hostlist
    hostlist_ports.append(int(hostspec_port))
ValueError: invalid literal for int() with base 10: 'my-password@my-db-server.postgres.database.azure.com:5432'

I see two problems here:

• the password leak in the logs (even tough I'm not sure how to avoid this in a consistent manner…)
• the fact that asyncpg incorrectly split the netloc part of the uri by splitting at the first @ intead of the last one.

The ticket is about the second problem:

• I think it should be safe to assume that everything before the last @ in the netloc is for authentication.
• Also by the way, it should also be safe to assume that everything after the first : in the auth part to be the password.

For the record, when using sqlalchemy + psycogs2, I have no problem using this kind of url to connect to my database.

[victornoel]

One more point: I tried using %40 instead of the @ but then it is sent directly as-is to the server which does not recognize the name.

[elprans]

This is already fixed in #472 and will be in the next release soon.

[victornoel]

@elprans great, thank you? And do you have an ETA for this next release btw?

[elprans]

Waiting on #295 to land, and also need to do some PostgreSQL 12 testing.
The release should happen in a few days.

[victornoel]

@elprans excellent, thx :)

[KrunchMuffin]

I appear to be getting this because asyncpg thinks there should be a port after the first colon, but it's the username:password colon.

    addrs, params = _parse_connect_dsn_and_args(
  File "C:\venv\lib\site-packages\asyncpg\connect_utils.py", line 237, in _parse_connect_dsn_and_args
    host, port = _parse_hostlist(dsn_hostspec, port, unquote=True)
  File "C:\venv\lib\site-packages\asyncpg\connect_utils.py", line 197, in _parse_hostlist
    hostlist_ports.append(int(hostspec_port))
ValueError: invalid literal for int() with base 10: 'abcde'

[nikolalazovski]

I am getting the same error for parsing postgresql://sql-module@gcp-project.iam:anystring@localhost:5432/app- ValueError: invalid literal for int() with base 10: 'anystring@localhost:5432', Also I tested with the connection string from the first post with all the asyncpg versions from 0.19-0.25 but I was getting the same error. Can anyone point what is the right version so that the connection string is interpreted well? Thanks

[elprans]

Those aren't valid DSNs, you must urllib.parse.quote the parts properly.