extmod/mbedtls: Implement DTLS HelloVerify, update test for MSG_PEEK

[projectgus]

Summary

• Updates the DTLS support initially added in extmod/modussl_mbedtls: Updated Wire in support for DTLS. #15764.
• DTLS 1.2 requires the HelloVerify protocol. This was already enabled on esp32 port (part of ESP-IDF config), and this PR enables it on bare metal ports as well as anti-replay attack protection.
• HelloVerify requires a session cookie store to be implemented in the application that calls mbedTLS. This function is added to the tls module SSLContext, and is transparent to Python code (provided the same SSLContext is reused).
• Add a config flag to disable DTLS on bare metal ports that don't need it. Enabled on "Extra" and above by default, so I think all ports with mbedTLS will get DTLS by default.
• Updated the multi_net test to work with HelloVerify and the new MSG_PEEK receive flag added in extmod/modlwip,esp32,unix: Add support for socket recv flags argument #17312. No longer needs an additional byte to be sent at the start of each connection.

TODO

• The current session cookie algorithm is too naive and doesn't achieve the purpose of HelloVerify (and can be used to DoS the server by creating too many cookies). It should be updated along the lines of the suggestion in the DTLS specification (Cookie = HMAC(Secret, Client-IP, Client-Parameters)).

Testing

• [*] Ran the new DTLS multi_net test on rp2, stm32 (PYBD_SF2) and esp32 ports. I'm still having (seemingly unrelated) issues with high outgoing UDP packet loss on rp2, but stm32 and esp32 ports passed the test in both directions.
• Re-run test with the new session cookie algorithm.
• Manually test the simple DTLS server provided with mbedTLS against a MicroPython client.

Trade-offs and Alternatives

• It's not clear how high the risk of MicroPython DTLS servers amplifying a DDoS attack is, so perhaps the HelloVerify support is not needed on the server side. However, connecting to a (compliant) DTLS server requires the DTLS client to implement HelloVerify, and mbedTLS (reasonably) treats HelloVerify as enabled for both server and client concurrently so we need to implement something on the server side.

This work was funded through GitHub sponsors.

[projectgus]

While reviewing this myself I realised the server DTLS cookie support I implemented is far too naive. Have marked as draft until I have time to improve it.

[codecov]

Codecov Report

Attention: Patch coverage is 86.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 98.53%. Comparing base (9bde125) to head (25d8871).

Files with missing lines	Patch %	Lines
extmod/modtls_mbedtls.c	86.66%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #17441      +/-   ##
==========================================
- Coverage   98.54%   98.53%   -0.01%     
==========================================
  Files         169      169              
  Lines       21941    21954      +13     
==========================================
+ Hits        21621    21632      +11     
- Misses        320      322       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

Code size report:

   bare-arm:    +0 +0.000% 
minimal x86:    +0 +0.000% 
   unix x64: +2256 +0.264% standard
      stm32:    +0 +0.000% PYBV10
     mimxrt:    +0 +0.000% TEENSY40
        rp2: +1240 +0.135% RPI_PICO_W
       samd:    +0 +0.000% ADAFRUIT_ITSYBITSY_M4_EXPRESS
  qemu rv32:    +0 +0.000% VIRT_RV32

[projectgus]

• The current session cookie algorithm is too naive and doesn't achieve the purpose of HelloVerify (and can be used to DoS the server by creating too many cookies). It should be updated along the lines of the suggestion in the DTLS specification (Cookie = HMAC(Secret, Client-IP, Client-Parameters)).

This is a lot harder than I initially thought, because in MicroPython modmbedtls works with an abstract stream and doesn't have any knowledge of the client IP. Also, adding this will probably add even more code size. But what's currently in this draft is woefully inadequate.

There's two possible ways forward:

1. Add another Python API extension, so the server code does something like s = ctx.wrap_socket(s, server_side=1, client_cookie=hmac_result) or s = ctx.dtls_wrap_socket(...). This offloads the responsibility (and code size cost) for calculating a secure cookie to Python code (and we could provide a sample implementation or a thin wrapper in micropython-lib that does it correctly). The server code can choose to pass a constant string here, at the cost of potentially becoming a DDoS vector if exposed to untrusted clients.
2. Drop DTLS server support, support DTLS client only. This makes testing a little harder as we'll need a DTLS server endpoint, but a simple no-op server should be easy enough to stand up (and safe to stand up on the public internet if it implements Hello Verify!)

Leaning towards (2) at the moment because I think the use case for DTLS client in MicroPython is much stronger than the case for needing a DTLS server, but that would mean removing functionality that was in v1.25 (albeit not fully compliant in that version). @keenanjohnson you implemented the original DTLS support in #15764, do you have any thought about this?

[dpgeorge]

Would it be simpler and more general to require that these functions be implemented in Python, and the user passes them through when wrapping the socket? Eg:

cookie_dict = {}
def dtls_cookie_handler(ctx, key, value):
    if value is None:
        # create and return new cookie
        cookie = os.urandom(32)
        cookie_dict[key] = cookie
        return cookie
    else:
        # check cookie
        return cookie_dict.get(key) == value
ctx.wrap_socket(..., dtls_cookie=dtls_cookie_handler)

[keenanjohnson]

In regards to your question @projectgus , I personally don't see a lot of use for the DTLS server on an embedded device. However, I do think that having the server implemented in micropython does make the unit testing vastly less brittle. Every time I have tied unit tests to some sort of live public service (like you suggest in option 2), I have personally regretted the choice in the long run as it tends to be brittle, people start to encounter all sorts of spurious local test failiures, etc etc.

Given that our main usecase for the server is just unit tests, I feel like moving the cookie calculation to python isn't the worst as long as we caveat the potential for a DDOS in the docs or something in case anyone ever comes along and has a real use case for the server.

Thanks for thinking of me and great work on the project as always.

[projectgus]

Thanks @dpgeorge and @keenanjohnson for your thoughts:

Alternative 1

I've pushed a version that uses the mbedTLS ssl_cookie implementation in C. All that the Python code needs to do is supply a client transport identifier (i.e. bytes representing IP & port) when it wraps the socket.

Code size cost is less bad than I thought: rp2: +1240. ESP32 impact will be a little lower as some of this was compiled in already. But still a lot of code size!

Alternative 2

Alternative to this, along the lines suggested by @dpgeorge above, would be to implement the cookie support in Python - but it probably still need to be a two step process in order to pass through the client's transport id:

cookie_dict = {}
def dtls_cookie_handler(ctx, key, value):
    encoded_cookie = binary encoding of (timestamp, HMAC(secret, key + timestamp)
    if value is None:
        return encoded_cookie
    else:
        # compare encoded_cookie matches `value` and also check the prepended timestamp isn't too old

ctx.set_dtls_cookie_handler(dtls_cookier_handler)

while True:
        _, client_addr = s.recvfrom(1, socket.MSG_PEEK)
        print("incoming connection")
        s.connect(client_addr)  # Connect back to the client

        # Wrap the UDP socket in server mode.
        try:
            s = ctx.wrap_socket(s, server_side=1, client_id=repr(client_addr).encode())

Alternative 3

There's a slightly hackier version where we pass it all through to wrap_socket() in one go, i.e.:

  client_id = repr(client_addr).encode()
  s = ctx.wrap_socket(s, server_side=1,
            cookie_handler= lambda ctx, key, value: dtls_client_handler(ctx, key, value, client_id))

... but the necessary implementation is hacky as would need to pass the mp_obj_t pointer via the client_id bytes and then decode it on the other side. If cookie_handler needs to stay valid after wrap_socket returns (unclear) then it'd need to also be stored somewhere in the ssl socket object so GC doesn't pick it up.