[feature] add non-exhausive URI validation

[yozachar]

• Succeeds:
  • feature request: validators.uri() #40,
  • Support for mailto: and tel: protocols ? #50,
  • Magnet validator #68 &
  • ipfs uri validation #195
  • URL Validation: Add support for other schemes apart from https, http, ftp #327

---

Suggestions are welcome:

validators/src/validators/uri.py

Lines 24 to 41 in cdc987d

	@validator
	def uri(value: str, /):
	"""Return whether or not given value is a valid URI.
	Examples:
	>>> uri('mailto:example@domain.com')
	# Output: True
	>>> uri('file:path.txt')
	# Output: ValidationError(func=uri, ...)
	Args:
	value:
	URI to validate.
	Returns:
	(Literal[True]): If `value` is a valid URI.
	(ValidationError): If `value` is an invalid URI.
	"""

[tofetpuzo]

are you taking any new suggestions?

[yozachar]

Constructive ones, please go ahead.

[tofetpuzo]

--- # Define a list of valid URI schemes => VALID_SCHEMES = ['http', 'https', 'ftp', 'mailto', 'file']

1. We could make the parameter-> value(str) flexible perhaps not restrict it to a type str, could be src/path/to/file; which could contain different urls
2. do a recursive search(regex expression perhaps) to match pattern

[tofetpuzo]

is it something we should do?

[yozachar]

Something similar yeah.

[tofetpuzo]

can I take up the challenge :)?

[yozachar]

Sure, go ahead.

[tofetpuzo]

Hey @yozachar ,can we use socket.inet_aton(ip) in python to validate ip addresses as a function then update the schema in uri.py?

   # URL-based schemes
    if any(
        value.startswith(item + "://") for item in {
            "ftp", "ftps", "git", "http", "https",
            "irc", "rtmp", "rtmps", "rtsp", "sftp",
            "ssh", "telnet", "gopher", "ldap", "sip",
            "nfs", "mqtt", "smb", "udp"
        }
    ):

[yozachar]

IP addresses are already validated in ip_address.py. Why use socket.inet_aton?

[tofetpuzo]

Hi, that is cool I looked at the message thread, something about this error below , was why I thought this feature was asked for I guess, see code below.

import validators

streamurl = "rtmp://192.168.1.123:1935/live/test"
print(validators.url(streamurl))

Output:
ValidationError(func=url, args={'value': 'rtmp://192.168.1.123:1935/live/test'})

Or does it mean we just want to validate other schemas? like the ones few ones below, that I could not find in uri.py

proposed schemas

"Uri":  {
"udp": udp://192.168.1.1:1234/path;param=value?,

"gopher": gopher://gopher.example.com/1/path;type=1?search#frag,\
"ldap": ldap://ldap.example.com/cn=John%20Doe,dc=example,dc=com;scope=one?sn#frag, \
"sip":sip://user:password@sip.example.com/path;transport=tcp?subject=Hello#frag, \
"smb": smb://fileserver.example.com/share/path;param=value?query=1#fragment\
 }

Few Thoughts

1. Have a regex expression : that complies the schema above.
2. uri_regrex: re.compile()
   e.g

  # Validate using regex for URL-based schemes
    if uri_regex.match(value):
        return True

[yozachar]

Apologies for the late reply.

Thank you for the inputs!

1. Exposing _validate_netloc and _validate_optionals (as url_netloc and url_optionals) would allow a reimplementation of url for schemes in application code, that would never make into the library

Currently, the url(...) validator is tightly coupled with these supporting functions. I assume exposing those functions would require further modifications..

2. Relaxing scheme check url(..., skip_scheme_check=True)

This function already has many parameters..

3. User supplied scheme list url(..., schemes: frozenset[str] = frozenset({"http", ...}) which is then passed to _validate_scheme

Please read #396 (comment)

Would one or more of the above be an acceptable change to the library API?

Yes, if the changes do not stiffen the codebase.

Is there some other, preferred solution, that would work around the fixed list of supported schemes?

I am thinking about some class based validations to make this more flexible, but ideas are much welcome.

[e3krisztian]

No worries, I had another idea, and implemented it in

• feat: allow custom URL scheme validation #409

It can be optionally further enhanced with providing alternative validate_scheme implementations, like publicly exposing the default one, and maybe some of these:

def accept_all_schemes(scheme):
    return True

def accept_iana_schemes(scheme):
    # https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml
    # I do not intend to, but someone might want to implement this as well
    ...

def accept_schemes(*accepted_schemes):
    def validate(scheme):
        return scheme in accepted_schemes
    return validate

which could be used like

validators.url(value, validate_scheme=accept_all_schemes)
validators.url(value, validate_scheme=accept_schemes("http", "https"))

[tofetpuzo]

Apologies, I have been "mia". Has this feature been implemented?

[yozachar]

Apologies, I have been "mia". Has this feature been implemented?

Nope.

#409 (comment)

[tofetpuzo]

I will work on it I have been super busy, I have started already.