BearSSL_Sessions.ino does not demonstrate faster connection with session

[DarioGHub]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• If there is a stack dump, I have decoded it.
• I have filled out all fields below.

Platform

• Hardware: [ESP-12]
• Core Version: [3.0.2]
• Development Env: [Arduino IDE]
• Operating System: [Windows]

Settings in IDE

• Module: [Nodemcu]
• Flash Mode: []
• Flash Size: [4MB]
• lwip Variant: [v2 Lower Memory]
• Reset Method: []
• Flash Frequency: [80Mhz]
• CPU Frequency: [80 & 160 Mhz]
• Upload Using: [SERIAL]
• Upload Speed: [115200]
• Debug port: [Disabled or Serial]
• Debug Level: [None or SSL+TLS_MEM+HTTP_CLIENT+HTTP_SERVER]

Example sketch does not demonstrate faster connection when "Connecting again with the initialized session..."

MCVE https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/examples/BearSSL_Sessions/BearSSL_Sessions.ino

Serial output:
Connecting without sessions...Trying: api.github.com:443...BSSL:_connectSSL: start connection...
Total time: 2786ms

Connecting with an uninitialized session...Trying: api.github.com:443...BSSL:_connectSSL: start connection...
Total time: 2681ms

Connecting with the just initialized session...Trying: api.github.com:443...BSSL:_connectSSL: start connection...
Total time: 2675ms

Connecting again with the initialized session...Trying: api.github.com:443...BSSL:_connectSSL: start connection...
Total time: 3137ms

[Bighoneypot]

@DarioGHub
Hi Dario, try with CPU Frequency:[160Mhz]
If you use Platformio Add this line: board_build.f_cpu = 160000000L
in Platformio.ini

[DarioGHub]

Ok, using Arduino IDE, chose tools > CPU Frequency: 160 MHz, then recompiled sketch. Generally sped up connections by about 30 percent, but still without a noticeably difference when using a session.

Connecting without sessions...Trying: api.github.com:443...
HTTP/1.1 200 OK ... Total time: 1940ms
Connecting with an uninitialized session...Trying: api.github.com:443
HTTP/1.1 200 OK ... Total time: 1958ms
Connecting with the just initialized session...Trying: api.github.com:443
HTTP/1.1 200 OK ... Total time: 1885ms
Connecting again with the initialized session...Trying: api.github.com:443
HTTP/1.1 200 OK ... Total time: 1891ms

Maybe I'm expecting too much. Is 50 ms the 'faster' connection to be demonstrated by using sessions in this example?

[Bighoneypot]

@DarioGHub
Hi Dario, please paste your code

[fsommer1968]

Hm, DarioGHub wrote in his initial post:
Example sketch does not demonstrate faster connection when "Connecting again with the initialized session..."

MCVE https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/examples/BearSSL_Sessions/BearSSL_Sessions.ino

[Bighoneypot]

Dear @fsommer1968 , yes I have see the first link in initial post, but my request is to see true code in use.

[DarioGHub]

Eliminated a couple of possible reasons for lack of faster connection with session by using Arduino IDE to enable/disable debugging.
Same results as mentioned above: 3 sec. @ 80 MHz, 2 sec. @ 160 MHz.
Updated original post; added Settings in IDE > Debug port & Debug Level

Does user code have to tell the library to reuse the session on subsequent connections?

Actual code was as follows:
Copied BearSSL_Sessions.ino Raw (19b7a29 Mar 3, 2022), and pasted in new sketch
Created New Tab (in Arduino IDE) named "certs.h"
Copied certs.h Raw (not pasted below for brevity), and pasted into new tab certs.h
Edited wlan/pass
Compiled @ 80 or 160 MHz, with or without debugging

// Example of using SSL sessions to speed up SSL connection initiation
//
// September 2018 by Earle F. Philhower, III
// Released to the public domain

#include <ESP8266WiFi.h>
#include <time.h>
#include "certs.h"

#ifndef STASSID
#define STASSID "A28"
#define STAPSK "passout"
#endif

const char *ssid = STASSID;
const char *pass = STAPSK;

const char *path = "/";

void setup() {
  Serial.begin(115200);
  Serial.println();
  Serial.println();

  Serial.printf("Connecting to %s\n", ssid);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, pass);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("\nConnected");
  Serial.println("IP Address: ");
  Serial.println(WiFi.localIP());

  // Set up time to allow for certificate validation
  configTime(3 * 3600, 0, "pool.ntp.org", "time.nist.gov");

  Serial.print("Waiting for NTP time sync: ");
  time_t now = time(nullptr);
  while (now < 8 * 3600 * 2) {
    delay(500);
    Serial.print(".");
    now = time(nullptr);
  }
  Serial.println("");
  struct tm timeinfo;
  gmtime_r(&now, &timeinfo);
  Serial.print("Current time: ");
  Serial.print(asctime(&timeinfo));
}

// Try and connect using a WiFiClientBearSSL to specified host:port and dump HTTP response
void fetchURL(BearSSL::WiFiClientSecure *client, const char *host, const uint16_t port, const char *path) {
  if (!path) { path = "/"; }

  Serial.printf("Trying: %s:443...", host);
  client->connect(host, port);
  if (!client->connected()) {
    Serial.printf("*** Can't connect. ***\n-------\n");
    return;
  }
  Serial.printf("Connected!\n-------\n");
  client->write("GET ");
  client->write(path);
  client->write(" HTTP/1.0\r\nHost: ");
  client->write(host);
  client->write("\r\nUser-Agent: ESP8266\r\n");
  client->write("\r\n");
  uint32_t to = millis() + 5000;
  if (client->connected()) {
    do {
      char tmp[32];
      memset(tmp, 0, 32);
      int rlen = client->read((uint8_t *)tmp, sizeof(tmp) - 1);
      yield();
      if (rlen < 0) { break; }
      // Only print out first line up to \r, then abort connection
      char *nl = strchr(tmp, '\r');
      if (nl) {
        *nl = 0;
        Serial.print(tmp);
        break;
      }
      Serial.print(tmp);
    } while (millis() < to);
  }
  client->stop();
  Serial.printf("\n-------\n\n");
}


void loop() {
  uint32_t start, finish;
  BearSSL::WiFiClientSecure client;
  BearSSL::X509List cert(cert_DigiCert_High_Assurance_EV_Root_CA);

  Serial.printf("Connecting without sessions...");
  start = millis();
  client.setTrustAnchors(&cert);
  fetchURL(&client, github_host, github_port, path);
  finish = millis();
  Serial.printf("Total time: %dms\n", finish - start);

  BearSSL::Session session;
  client.setSession(&session);
  Serial.printf("Connecting with an uninitialized session...");
  start = millis();
  client.setTrustAnchors(&cert);
  fetchURL(&client, github_host, github_port, path);
  finish = millis();
  Serial.printf("Total time: %dms\n", finish - start);

  Serial.printf("Connecting with the just initialized session...");
  start = millis();
  client.setTrustAnchors(&cert);
  fetchURL(&client, github_host, github_port, path);
  finish = millis();
  Serial.printf("Total time: %dms\n", finish - start);

  Serial.printf("Connecting again with the initialized session...");
  start = millis();
  client.setTrustAnchors(&cert);
  fetchURL(&client, github_host, github_port, path);
  finish = millis();
  Serial.printf("Total time: %dms\n", finish - start);

  delay(10000);  // Avoid DDOSing github
}

[Bighoneypot]

@DarioGHub
this is my result

12:19:45.831 -> Connected
12:19:45.831 -> IP Address: 
12:19:45.831 -> 192.168.137.XXX
12:19:45.831 -> Waiting for NTP time sync: .
12:19:46.343 -> Current time: Fri Mar 18 11:19:46 2022
12:19:46.343 -> Connecting without sessions...Trying: api.github.com:443...Connected!
12:19:46.944 -> -------
12:19:47.269 -> HTTP/1.1 403 rate limit exceeded
12:19:47.269 -> -------
12:19:47.269 -> 
12:19:47.269 -> Total time: 903ms
12:19:47.269 -> Connecting with an uninitialized session...Trying: api.github.com:443...Connected!
12:19:47.876 -> -------
12:19:48.202 -> HTTP/1.1 403 rate limit exceeded
12:19:48.202 -> -------
12:19:48.202 -> 
12:19:48.202 -> Total time: 929ms
12:19:48.202 -> Connecting with the just initialized session...Trying: api.github.com:443...Connected!
12:19:48.762 -> -------
12:19:49.042 -> HTTP/1.1 403 rate limit exceeded
12:19:49.042 -> -------
12:19:49.042 -> 
12:19:49.042 -> Total time: 859ms
12:19:49.042 -> Connecting again with the initialized session...Trying: api.github.com:443...Connected!
12:19:49.601 -> -------
12:19:49.927 -> HTTP/1.1 403 rate limit exceeded
12:19:49.927 -> -------
12:19:49.927 -> 
12:19:49.927 -> Total time: 865ms

Basic SSL ciphers.

[DarioGHub]

Thanks @Bighoneypot for testing. What conclusion do you draw from the above results?
IMHO, your results, like mine, do NOT show an improvement when using a session.

[Bighoneypot]

@DarioGHub
At the moment the improvement is present, but it is not a major improvement. I'll do a few more tests this week if possible.

[earlephilhower]

You are not seeing a difference because you're not actually connecting with the same HTTPS server. api.github.com lives behind a massive load balancer, so successive connections will end up talking to different machines. Those machines do not share TLS session information (this is not cookie-like), so you see no difference.

Changing the URL to a machine I own which is not behind a load balancer (www.ziplabel.com, also need to update the cert or use client.setInsecure() before each connection attempt) shows a massive difference (>50% speedup) in connect speeds:

Connecting without sessions...Trying: www.ziplabel.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 1228ms
Connecting with an uninitialized session...Trying: www.ziplabel.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 1146ms
Connecting with the just initialized session...Trying: www.ziplabel.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 509ms
Connecting again with the initialized session...Trying: www.ziplabel.com:443...Connected!
-------
HTTP/1.1 200 OK
-------

Total time: 509ms

[DarioGHub]

Thank you @earlephilhower for your many contributions, and pointing out the limitation of sessions when used against multiple servers masquerading as one, and the limitation of my imagination. In hindsight, of course I should not have relied so much on the example, and changed the host just like we change passwords. Does a note to this effect belong in the example?

[earlephilhower]

Fair enough, @DarioGHub . I've got a PR in with a connection to the ESP8266.com forum which shows a 50% speedup consistently, with some comments in the INO about the limits of sessions.