probeMaxFragmentLength of BearSSL::WiFiClientSecure is not compatible with OpenSSL supporting Maximum Fragment Length Negotiation extension

[sislakd]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: ESP-12
• Core Version: 2c36cfe
• Development Env: Arduino IDE
• Operating System: MacOS

Problem Description

The method probeMaxFragmentLength of BearSSL::WiFiClientSecure is not compatible with OpenSSL supporting Maximum Fragment Length Negotiation extension. I've tested on OpenSSL 1.1.1b1 (beta 1) and also the latest OpenSSL 1.1.1 build. The method probeMaxFragmentLength generates very simple ClientHello message which is rejected by the server with the following error:

Client connection from X.X.X.X failed: error:1417A0C1:SSL routines:tls_post_process_client_hello:no shared cipher.

The server drops connection immediately during processing of ClientHello message. OpenSSL 1.1.1 supports usage of Maximum Fragment Length Negotiation extension properly. If probing is not done but setBufferSizes is used before connection, server accepts extension sent in ClientHello and confirms selected max fragment length in ServerHello.

MCVE Sketch

Use example in libraries/ESP8266WiFi/examples/BearSSL_MaxFragmentLength/BearSSL_MaxFragmentLength.ino against a server backed by OpenSSL 1.1.1 (Mosquitto, Haproxy or Apache). The function fetchMaxFragmentLength in this example will always report that MFLN is not supported even it is working well. If you skip probing, setBufferSizes(512, 512) and examine ClientHello and ServerHello packets, you will find that server accepts MFLN and fragments are not bigger than 512.

[Jeroen88]

I use the latest Node.js release as backend with the Standard OpenSSL and that works like a charm

[earlephilhower]

@sislakd, you'll need to provide some example code. I've just tried with a locally built 1.1.1a on Ubuntu and MFLN negotiation reports success:

earle@server:~/src/openssl-1.1.1a$ LD_LIBRARY_PATH=/home/earle/src/openssl-1.1.1a:$LD_LIBRARY_PATH apps/openssl s_server -debug -state -key key.pem -cert cert.pem -accept 44331 -WWW
Using default temp DH parameters
ACCEPT
SSL_accept:before SSL initialization
read from 0x5599e92a9950 [0x5599e92b5573] (5 bytes => 5 (0x5))
0000 - 16 03 03 00 8c                                    .....
read from 0x5599e92a9950 [0x5599e92b5578] (140 bytes => 140 (0x8C))
0000 - 01 00 00 88 03 03 01 02-03 04 05 06 07 08 11 12   ................
0010 - 13 14 15 16 17 18 01 02-03 04 05 06 07 08 11 12   ................
0020 - 13 14 15 16 17 18 00 00-5a cc a9 cc a8 c0 2b c0   ........Z.....+.
0030 - 2f c0 2c c0 30 c0 ac c0-ad c0 ae c0 af c0 23 c0   /.,.0.........#.
0040 - 27 c0 24 c0 28 c0 09 c0-13 c0 0a c0 14 c0 2d c0   '.$.(.........-.
0050 - 31 c0 2e c0 32 c0 25 c0-29 c0 26 c0 2a c0 04 c0   1...2.%.).&.*...
0060 - 0e c0 05 c0 0f 00 9c 00-9d c0 9c c0 9d c0 a0 c0   ................
0070 - a1 00 3c 00 3d 00 2f 00-35 c0 08 c0 12 c0 03 c0   ..<.=./.5.......
0080 - 0d 00 0a 01 00 00 05 00-01 00 01 01               ............
SSL_accept:before SSL initialization
SSL_accept:SSLv3/TLS read client hello
SSL_accept:SSLv3/TLS write server hello
SSL_accept:SSLv3/TLS write certificate
write to 0x5599e92a9950 [0x5599e92c2ce0] (983 bytes => 983 (0x3D7))
0000 - 16 03 03 00 51 02 00 00-4d 03 03 49 63 94 e2 b5   ....Q...M..Ic...
0010 - 3b 1b 7c 10 6f c8 1b 21-4c 01 53 3e e4 e7 f7 7e   ;.|.o..!L.S>...~
0020 - c0 a4 81 44 4f 57 4e 47-52 44 01 20 60 13 8b 6c   ...DOWNGRD. `..l
0030 - e3 28 24 5f 31 5c ca a3-91 6b ba af 4b e7 56 39   .($_1\...k..K.V9
0040 - 09 59 ad 42 31 77 c5 fb-90 7a b4 5d 00 9c 00 00   .Y.B1w...z.]....
0050 - 05 00 01 00 01 01 16 03-03 02 00 0b 00 03 6a 00   ..............j.
0060 - 03 67 00 03 64 30 82 03-60 30 82 02 48 a0 03 02   .g..d0..`0..H...
0070 - 01 02 02 09 00 95 99 e2-3c f1 52 2d a9 30 0d 06   ........<.R-.0..
0080 - 09 2a 86 48 86 f7 0d 01-01 0b 05 00 30 45 31 0b   .*.H........0E1.
0090 - 30 09 06 03 55 04 06 13-02 41 55 31 13 30 11 06   0...U....AU1.0..
00a0 - 03 55 04 08 0c 0a 53 6f-6d 65 2d 53 74 61 74 65   .U....Some-State
00b0 - 31 21 30 1f 06 03 55 04-0a 0c 18 49 6e 74 65 72   1!0...U....Inter
00c0 - 6e 65 74 20 57 69 64 67-69 74 73 20 50 74 79 20   net Widgits Pty 
00d0 - 4c 74 64 30 1e 17 0d 31-39 30 34 31 37 32 30 34   Ltd0...190417204
00e0 - 32 33 39 5a 17 0d 32 30-30 34 31 36 32 30 34 32   239Z..2004162042
00f0 - 33 39 5a 30 45 31 0b 30-09 06 03 55 04 06 13 02   39Z0E1.0...U....
0100 - 41 55 31 13 30 11 06 03-55 04 08 0c 0a 53 6f 6d   AU1.0...U....Som
0110 - 65 2d 53 74 61 74 65 31-21 30 1f 06 03 55 04 0a   e-State1!0...U..
0120 - 0c 18 49 6e 74 65 72 6e-65 74 20 57 69 64 67 69   ..Internet Widgi
0130 - 74 73 20 50 74 79 20 4c-74 64 30 82 01 22 30 0d   ts Pty Ltd0.."0.
0140 - 06 09 2a 86 48 86 f7 0d-01 01 01 05 00 03 82 01   ..*.H...........
0150 - 0f 00 30 82 01 0a 02 82-01 01 00 dc 6f 0e 60 67   ..0.........o.`g
0160 - 4d 4e de dc cf 1b 62 97-fc ea 16 c5 16 a8 2f 9b   MN....b......./.
0170 - 19 18 56 3c 2c 5a de 65-83 38 41 ce c4 ed cc 47   ..V<,Z.e.8A....G
0180 - af 8c bb 57 36 2a 44 a9-2e b5 5d 38 d6 b4 d9 89   ...W6*D...]8....
0190 - fc b0 47 65 89 9d 9c 57-77 d6 97 20 af 00 e8 b5   ..Ge...Ww.. ....
01a0 - 20 ea b5 6b 8c 61 21 32-f5 23 7b a0 95 0d da b7    ..k.a!2.#{.....
01b0 - 3d 5f 63 20 08 6e 0d 85-12 61 4d e6 f8 09 ef a3   =_c .n...aM.....
01c0 - 5a ef a7 a0 7f 4e 16 98-79 b6 7b b0 f7 6d fb 7c   Z....N..y.{..m.|
01d0 - 43 45 0e 66 64 e4 f2 6e-45 df 42 d8 66 5c 12 c1   CE.fd..nE.B.f\..
01e0 - af dd 3e ef 4d 67 73 61-22 38 2b f2 6a e5 d1 06   ..>.Mgsa"8+.j...
01f0 - 98 29 66 47 cc b4 8b b2-5f 0f aa 37 14 4a 22 a6   .)fG...._..7.J".
0200 - 6f ad f4 05 ac da c2 05-8e 48 2d 3d 1c 64 6a a5   o........H-=.dj.
0210 - 7b d0 a6 bc 68 ba 7f 06-ec 1e f2 c4 78 6f 05 97   {...h.......xo..
0220 - bc 78 5a 83 09 96 c4 d9-36 f7 07 19 79 99 ba a8   .xZ.....6...y...
0230 - ef 23 a0 8e 07 ab c8 5d-35 20 e9 00 60 cc b7 ca   .#.....]5 ..`...
0240 - e2 c1 1a b6 9c 11 54 9f-de a0 c9 6b 10 bf 56 40   ......T....k..V@
0250 - 8b a9 d0 7b a0 df d0 25-ab d4 77 16 03 03 01 6e   ...{...%..w....n
0260 - 02 03 01 00 01 a3 53 30-51 30 1d 06 03 55 1d 0e   ......S0Q0...U..
0270 - 04 16 04 14 7c 43 74 67-38 ba 9c b6 94 7e 7f 26   ....|Ctg8....~.&
0280 - 68 48 75 18 42 53 2f 93-30 1f 06 03 55 1d 23 04   hHu.BS/.0...U.#.
0290 - 18 30 16 80 14 7c 43 74-67 38 ba 9c b6 94 7e 7f   .0...|Ctg8....~.
02a0 - 26 68 48 75 18 42 53 2f-93 30 0f 06 03 55 1d 13   &hHu.BS/.0...U..
02b0 - 01 01 ff 04 05 30 03 01-01 ff 30 0d 06 09 2a 86   .....0....0...*.
02c0 - 48 86 f7 0d 01 01 0b 05-00 03 82 01 01 00 a9 0b   H...............
02d0 - 12 63 75 6e 30 1c 2e e0-ec 9a ad 70 33 35 50 cd   .cun0......p35P.
02e0 - 25 22 49 b9 01 70 42 75-a1 ee 09 11 47 90 21 6c   %"I..pBu....G.!l
02f0 - d2 b7 c6 8d ca 83 1c c6-44 58 1f 58 66 76 88 cd   ........DX.Xfv..
0300 - 98 5f bb c0 70 06 a7 bf-02 4f 2a e0 48 27 92 25   ._..p....O*.H'.%
0310 - 19 d7 17 5b f2 59 e1 9c-cc a8 9a be 32 af 4b 8d   ...[.Y......2.K.
0320 - f5 a7 f0 fe a9 d9 03 c5-5c 09 2b 4d 46 92 da 05   ........\.+MF...
0330 - 83 68 5e a6 22 e6 9f 72-94 12 90 86 a5 ac 41 f5   .h^."..r......A.
0340 - a5 b9 6f 7a d8 53 2b 9e-0c 86 3e 54 f6 46 7b 3e   ..oz.S+...>T.F{>
0350 - 62 34 02 1f 16 f9 73 32-00 99 0b 84 b8 80 6f c2   b4....s2......o.
0360 - 74 44 87 cb 15 c0 3e 51-c1 4c 37 52 92 af 9f 09   tD....>Q.L7R....
0370 - d8 a5 26 86 35 da 58 85-8b aa 52 b5 46 0e 91 d0   ..&.5.X...R.F...
0380 - 2d 2e bb f8 4a a1 21 25-79 35 00 13 dc 71 4d e8   -...J.!%y5...qM.
0390 - 29 4d bd e5 fa fd 72 85-2f 39 33 52 1a fc 86 f8   )M....r./93R....
03a0 - b4 9e 8e f5 fc d4 e8 6d-f6 06 74 f0 5c 9e d0 af   .......m..t.\...
03b0 - 79 5e f1 16 26 e7 7b 7a-dd 49 8b 27 a3 5c 97 9d   y^..&.{z.I.'.\..
03c0 - 54 df 63 50 fe c8 54 a0-7d e2 1a 2e 6f 3a 16 03   T.cP..T.}...o:..
03d0 - 03 00 04 0e 00 00 00                              .......
SSL_accept:SSLv3/TLS write server done
read from 0x5599e92a9950 [0x5599e92b5573] (5 bytes => 5 (0x5))
0000 - 15 03 03 00 02                                    .....
read from 0x5599e92a9950 [0x5599e92b5578] (2 bytes => 2 (0x2))
0000 - 01 5a                                             .Z
SSL3 alert read:warning:user canceled
read from 0x5599e92a9950 [0x5599e92b5573] (5 bytes => 5 (0x5))
0000 - 15 03 03 00 02                                    .....
read from 0x5599e92a9950 [0x5599e92b5578] (2 bytes => 2 (0x2))
0000 - 01 00                                             ..
SSL3 alert read:warning:close notify
SSL_accept:error in SSLv3/TLS write server done

On the client I made setup:

void setup() {
  Serial.begin(115200);

  delay(1000);
  Serial.println();
  Serial.println();

  // We start by connecting to a WiFi network
  Serial.print("Connecting to ");
  Serial.print(ssid);

  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, pass);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");

  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
  while(1) {
    WiFiClientSecure client;
      bool mfln = client.probeMaxFragmentLength("192.168.1.8", 44331, 512);
      Serial.printf("%s\n", mfln?"ok":"fail");
      delay(1000);
  }
}

And it always reports "OK" (so MLFN scanning worked).

[earlephilhower]

The "user cancelled" bit of the protocol is expected, that's the way the probing works. Once it gets the header back saying the MFLN was accepted it stops the connection attempt.

[sislakd]

Here is output from my test:

Using default temp DH parameters
ACCEPT
SSL_accept:before SSL initialization
read from 0x1c298f8 [0x1c34d7b] (5 bytes => 5 (0x5))
0000 - 16 03 03 00 34                                    ....4
read from 0x1c298f8 [0x1c34d80] (52 bytes => 52 (0x34))
0000 - 01 00 00 30 03 03 01 02-03 04 05 06 07 08 11 12   ...0............
0010 - 13 14 15 16 17 18 01 02-03 04 05 06 07 08 11 12   ................
0020 - 13 14 15 16 17 18 00 00-02 cc a8 01 00 00 05 00   ................
0030 - 01 00 01 02                                       ....
SSL_accept:before SSL initialization
write to 0x1c298f8 [0x1c43510] (7 bytes => 7 (0x7))
0000 - 15 03 03 00 02 02 28                              ......(
SSL3 alert write:fatal:handshake failure
SSL_accept:error in error
1996079120:error:1417A0C1:SSL routines:tls_post_process_client_hello:no shared cipher:../ssl/statem/statem_srvr.c:2263:

Does your probeMaxFragmentLength expecting usage of legacy cipher suites (not Forward Secrecy)? I'm enforcing TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 for its Forward Secrecy and good performance of CHACHA20 on ESP8266. Maybe in that case ClientHello should include some additional components which are not sent by probeMaxFragmentLength at the moment (including only TLS version, Random, SessionID, Cipher Suites, No compression and MFLN extension). The full connection made by BearSSL using this cipher suite includes beside these also Signature algorithms extension (rsa_pkcs1_sha256), Supported Groups extension (secp256r1, secp384r1, secp521r1) and EC Point formats extension (uncompressed) in Client Hello. I think that ECDHE requires these otherwise server don't know how to generate ECDH server params for Server Key Exchange response immediately following Server Hello.

[earlephilhower]

Have you modifies the BearSSL /WiFiClientSecure code in any way? What you're showing the 8266 sending can't be possible otherwise.

0027,0028 == length of cipher list in bytes

In yours, it's showing "0002" meaning a single cipher. In mine, and in git head it shows "005a", meaning a bunch.

The code takes the size of a constant array and stuffs it into the packet followed by the contents of the constant array:

  clientHello[sizeof(clientHelloHead_P) + 0] = sizeof(suites_P) >> 8;   // MSB byte len
  clientHello[sizeof(clientHelloHead_P) + 1] = sizeof(suites_P) & 0xff; // LSB byte len
  for (size_t i = 0; i < sizeof(suites_P) / sizeof(suites_P[0]); i++) {
    uint16_t flip = pgm_read_word(&suites_P[i]);
    // Swap to network byte order
    flip = ((flip >> 8) & 0xff) | ((flip & 0xff) << 8);
    memcpy(clientHello + sizeof(clientHelloHead_P) + 2 + 2 * i, &flip, 2);
  }

So there's no way for 2 to find its way into the handshake packet unless sizeof(suites_P) != 0x5a.

[sislakd]

As mentioned I'm enforcing TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 for its Forward Secrecy and good performance of CHACHA20 on ESP8266. Thus, the modification is in used default suites_P .

[earlephilhower]

That's not the way to do it. Just set a custom list using the built in function setCiphers and pass in your list before connecting. The server expects a list of possible ciphers, in order or preference by the client. Your modified the core code is sending only a single cipher that OpenSSL doesn't support and hence the issue. Probing can and should send all ciphers in the world, since it never gets that far in the handshake process to actually select or use one.

Closing as not core related.

[sislakd]

TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 cipher is supported by OpenSSL as the connection made by BearSSL using this cipher suite runs well. The issue is the same if you sent all ciphers from probeMaxFragmentLength and target server is configured to accept only strong cipher suites on TLS1.2 (those which have been selected for TLS1.3). I've did experimental modification of probing function to include other necessary extensions and it works well now. Thus definitely, probing can be fixed.

The MFLN probing in the current form introduce security vulnerability. There is a security risk in having MFLN probing without verification of server identity. An attacker can accept usage of small fragment length being in the middle of the connection as a response to probeMaxFragmentLength Client Hello. Then an attacker will let real connection go to target server which is not supporting MFLN extension. This results into crash of client because server sends 16k fragments instead of requested small ones. The right way is to integrate MFLN verification directly into BearSSL where full server identity is verified (unless going with insecure mode). If configured receive buffer size is not accepted by the server with verified identity, connection should be terminated and calling code have to receive appropriate error code. Upon this error, client should enlarge receive buffer and try to make connection again. Optionally, the whole procedure can be done inside BearSSL transparently enlarging receive buffer size.

[earlephilhower]

Interesting observation, I didn't think of that particular scenario. Thanks for describing it.

In any case, the proper thing is to have the probe use the current cipher list, whether its the default (which you should not change) or one you set via setCiphers which can contain a single entry.

What is your exact OpenSSL 1.1.1 openssl s_server command line, so I can give a like to like comparison?

The SSL handshake protocol is rigid in the first few messages, and we only get as fas as the ClientHello/ServerHello response, so assuming the server accepts 0xcca8 as a cipher, I'm not sure where the trouble lies and need to see a live example to troubleshoot.

[devyte]

Also, @sislakd said:

I've did experimental modification of probing function to include other necessary extensions and it works well now

What exactly did you modify in the probing function?

[earlephilhower]

On second thought, there's a more basic issue. If you don't go through a complete handshake and cert verification then the probing as-is is still subject to MITM attack. So there is no need to change the cipher list in the probe call, it doesn't buy you anything.

Anything less than the full, 5-second+ SSL negotiation where certs are examined seems to suffer the same issue, so it's a matter of adding a new probeDeep() to check this. Not an insignificant change and will require 1K RAM (512b each buffer) to run.