OTA update with HTTPS is not working

[waghekapil]

Basic Infos

Hardware

Hardware: ?ESP8266-12E?
Core Version: ?2.4.0?

Description

OTA update without SSL is working fine. With SSL is it throwing error.

I generates FP through following command.

openssl x509 -noout -fingerprint -sha1 -inform pem -in C:\HTTPS\sipl.crt

Settings in IDE

Module: ?Generic ESP8266 Module?
Flash Size: ?4MB(3M SPIFFS)?
CPU Frequency: ?80Mhz?
Flash Mode: ?DIO?
Flash Frequency: ?40Mhz?
Upload Using: ?SERIAL?
Reset Method: ?ck?
Arduino SDK: ?1.8.5?

Sketch

/*
Name:    OTA.ino
Created: 2/13/2018 11:57:55 AM
Author:  kapil
*/

#include <Arduino.h>
#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>
#include <ESP8266httpUpdate.h>

void setup() {
	Serial.begin(115200);
	Serial.setDebugOutput(true);
	Serial.println("Hussain...");

	WiFi.begin("SSID", "PASS");
	while (WiFi.status() != WL_CONNECTED) {
		delay(1000);
	}
	Serial.println("Connected, updating");
	delay(1000);

	t_httpUpdate_return ret = ESPhttpUpdate.update("https://13.127.174.106:443/Blink.ino.bin", "", "07 CB 9F 52 17 54 4C 5D 84 93 31 49 F7 9A E0 43 69 FB 49 03");

	Serial.println();
	Serial.print("OTA free heap : ");
	Serial.print(ESP.getFreeHeap());

	Serial.println(F("\n---- ota update done ----\n"));
	switch (ret)
	{
	case HTTP_UPDATE_FAILED:
		Serial.printf("HTTP_UPDATE_FAILD Error (%d): %s", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
		break;
	case HTTP_UPDATE_NO_UPDATES:
		Serial.println(F("HTTP_UPDATE_NO_UPDATES"));
		break;
	case HTTP_UPDATE_OK:
		Serial.println(F("HTTP_UPDATE_OK"));
		break;
	default:
		Serial.printf("Unexpected response code %d from ESPhttpUpdate.update\n", (int)ret);
		break;

	}
}
void loop() {
}

Debug Messages

scandone
scandone
state: 0 -> 2 (b0)
state: 2 -> 3 (0)
state: 3 -> 5 (10)
add 0
aid 1
cnt

connected with SIPLEC-IOT, channel 1
dhcp client start...
ip:192.168.100.5,mask:255.255.255.0,gw:192.168.100.2
Connected, updating
State: sending Client Hello (1)
State: receiving Server Hello (2)
State: receiving Certificate (11)
=== CERTIFICATE ISSUED TO ===
Common Name (CN): S
Organization (O): SYSTEMATIX INFOTECH PVT LTD
Organizational Unit (OU): SIPL
Location (L): INDORE
Country (C): IN
State (ST): MP
Basic Constraints: CA:TRUE, pathlen:10000
=== CERTIFICATE ISSUED BY ===
Common Name (CN): S
Organization (O): SYSTEMATIX INFOTECH PVT LTD
Organizational Unit (OU): SIPL
Location (L): INDORE
Country (C): IN
State (ST): MP
Not Before: Tue Feb 20 09:36:08 2018
Not After: Wed Feb 20 09:36:08 2019
RSA bitsize: 2048
Sig Type: SHA256
State: receiving Server Hello Done (14)
State: sending Client Key Exchange (16)
State: sending Finished (16)
State: receiving Finished (16)
cert FP: F4 9C BE 82 ED 63 F3 D9 F3 8A 60 1D 9C 40 FE 06 8D C4 47 F7
test FP: 07 CB 9F 52 17 54 4C 5D 84 93 31 49 F7 9A E0 43 69 FB 49 03

OTA free heap : 38648
---- ota update done ----

HTTP_UPDATE_FAILD Error (-1): HTTP error: connection refused

[kristinpaget]

Same behaviour here. I've tried all the overloaded forms of ESPhttpUpdate.update and they all fail in various ways. The sketch below will:

• Fetch the HTTPS update URL with an HTTPClient first, just to make sure it works
• Attempts an update over SSL, which claims connection refused from the same URL
• Attempts the same update over plaintext, demonstrating that the server is working OK.

Sketch

#include "ESP8266WiFi.h"
#include "gpio.h"
#include "WiFiClientSecure.h"
#include "ESP8266httpUpdate.h"

#define LOG Serial.print

const char HTTPURL[] = "<something>";
const char HTTPSURL[] = "<something with an extra S>";
const char Fingerprint[] = "<An appropriate fingerprint>";

const char SSID[] = "SSID";
const char Pass[] = "Password";

void setup() 
{

  Serial.begin(115200, SERIAL_8N1);
  delay(2000); // Allow the other end to attach

  WiFi.mode(WIFI_STA);
  WiFi.begin(SSID, Pass);

  while (WiFi.status() != WL_CONNECTED)
  {
    LOG(".");
    delay(100);
  }
        
  LOG("Wifi connected!\r\n");
  LOG("My IP is ");
  LOG(WiFi.localIP());
  LOG("\r\n");

  gpio_init();
  pinMode(0,INPUT_PULLUP);

  LOG("Ready to go!\r\n");
}

void loop() 
{
  unsigned char Buffer[1024];
  HTTPClient Client;

  if (digitalRead(0) == HIGH)
  {
    // Waiting for button
    delay(100);
    return;
  }

  // Start the sequence.
  Client.begin(HTTPSURL, Fingerprint);

  LOG("About to GET ");
  LOG(HTTPSURL);
  LOG("\r\n");
  int Result = Client.GET();

  if(Result > 0)
  {
    sprintf((char *)Buffer, "TLS URL fetched!\r\nHTTP response code: %u\r\n%u bytes received\r\n", Result, Client.getSize());
    LOG((char *)Buffer);
  }
  else
  {
    LOG("HTTP request failed!\r\nError: ");
    LOG(Result);
    LOG("\r\n");
  }

  // Check for a firmware update
  LOG("Checking for firmware, same URL.\r\n");
  Result = ESPhttpUpdate.update(HTTPSURL, Fingerprint);

  // if successful, ESP will restart
  LOG("Oh noes, update failed!\r\nResult:");
  LOG(Result);
  LOG("\r\n");

  switch(Result) {
    case HTTP_UPDATE_FAILED:
        sprintf((char *)Buffer, "Update failed, error %d: %s\r\n", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
        LOG((char *)Buffer);
        break;
    case HTTP_UPDATE_NO_UPDATES:
        LOG("No update available.\r\n");
        break;
    case HTTP_UPDATE_OK:
        LOG("You shouldn't see this message because I should be rebooting :)\r\n");
        break;
  }  

  // Try again in plaintext...
  LOG("Checking for firmware insecurely...\r\n");
  LOG("Updating from ");
  LOG(HTTPURL);
  LOG("\r\n");

  Result = ESPhttpUpdate.update(HTTPURL);
  
  // if successful, ESP will restart
  LOG("Oh noes, update failed!\r\nResult:");
  LOG(Result);
  LOG("\r\n");

  switch(Result) {
    case HTTP_UPDATE_FAILED:
        sprintf((char *)Buffer, "Update failed, error %d: %s\r\n", ESPhttpUpdate.getLastError(), ESPhttpUpdate.getLastErrorString().c_str());
        LOG((char *)Buffer);
        break;
    case HTTP_UPDATE_NO_UPDATES:
        LOG("No update available.\r\n");
        break;
    case HTTP_UPDATE_OK:
        LOG("You shouldn't see this message because I should be rebooting :)\r\n");
        break;
  }  
}

Debug output:

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

load 0x4010f000, len 1384, room 16 
tail 8
chksum 0x2d
csum 0x2d
v4ceabea9
@cp:0
ld
......................Wifi connected!
My IP is 192.168.30.4
Ready to go!
About to GET <SSL URL>
TLS URL fetched!
HTTP response code: 200
344064 bytes received
Checking for firmware, same URL.
[httpUpdate] HTTP error: connection refused
Oh noes, update failed!
Result:0
Update failed, error -1: HTTP error: connection refused
Checking for firmware insecurely...
Updating from <plaintext URL>
[httpUpdate] Header read fin.
[httpUpdate] Server header:
[httpUpdate]  - code: 200
[httpUpdate]  - len: 344064
[httpUpdate] ESP8266 info:
[httpUpdate]  - free Space: 684032
[httpUpdate]  - current Sketch Size: 344064
[httpUpdate] runUpdate flash...
sleep disable
pm open,type:0 0
[httpUpdate] Update ok
state: 5 -> 0 (0)
rm 0
pm close 7
del if0
usl

 ets Jan  8 2013,rst cause:2, boot mode:(3,6)

[lucasromeiro]

I have a same problem.... Anyone solve this??

[earlephilhower]

Is this the same as #3908 or #4501?

One has to do with an out of memory error during HTTPS update (3908), while the other has found a problem in axTLS where the SSL handshake is expecting that each message is sent in a separate TCP packet but some servers pack multiple SSL handshake messages into a single fragment (4501).

Can you connect to the specified HTTPS sites using a simple WiFiClientSecure::connect? If not then I'd think it was #4501.

[aderusha]

Has anyone made this work successfully? I'm seeing the same behavior using the stock httpUpdate example against https://raw.githubusercontent.com.

[lucasromeiro]

I have not yet encaged a way to do OTA updates using HTTPS. nothing worked.

[me-no-dev]

I would say that 8266 does not have the memory to handle a whole firmware update over SSL. If you can devise a way to get the update in 4K chunks for example, it could work. Though it might take quite a bit longer...

[aderusha]

That's pretty much what I had expected. I've implemented a ridiculous workaround by way of a reverse proxy that will deliver the binary from GitHub over plain HTTP.

[earlephilhower]

There is a PR for a different SSL which doesn't need dynamic memory allocation after it's initiated (#4273) and supports the http update (you need to pass in a uint8_t[20] instead of a hex string to the constructor and everything else is identical as far as the updater API).

It may not work in your case, but it may be worth a try to avoid the workaround you've had to implement...

[lucasromeiro]

Has anyone managed to make it work ?? how to do this using http proxy?

[aderusha]

@lucasromeiro my case is pretty simple and the solution I've applied is straightforward. I've spun up an Ubuntu server in Azure, deployed NGINX, registered haswitchplate.com, pointed it at this host, and I've applied the following configuration:

server {
    listen 80;
    server_name haswitchplate.com;

    location /HASwitchPlate.ino.d1_mini.bin {
        proxy_pass https://raw.githubusercontent.com/aderusha/HASwitchPlate/master/Arduino_Sketch/HASwitchPlate.ino.d1_mini.bin;
    }
    location / {
        return 301 https://github.com/aderusha/HASwitchPlate;
    }
}

When a user requests http://haswitchplate.com/HASwitchPlate.ino.d1_mini.bin NGINX will proxy the request for the user, so it will connect to GitHub, start downloading https://raw.githubusercontent.com/aderusha/HASwitchPlate/master/Arduino_Sketch/HASwitchPlate.ino.d1_mini.bin, and hand that over to the connected user via HTTP. Any other URL is redirected to my GitHub repo for this project.

I've also put CloudFlare in front of this to cache the HASwitchPlate.ino.d1_mini.bin file in their CDN to improve global response time and cut down on the bandwidth consumption on my cloud server.

[apicquot]

If ESP8266's memory is too limited to work with SSL, it may useful to add in the httpUpdate an option to sign the binary or even fully crypt it? Connection may not be secure but at least the file could be authenticated.

[mischmerz]

Ok guys - still not working. My server only "speaks" https, I added the required ciphers and

[lucasromeiro]

I could not do it either. I think I will have to create another domain with another server that is http to be able to do the update ota ... because the current library can not manage the memoia.

[mischmerz]

It seems to be working now. My server only speaks https, I added the required codes and test the connection:

WiFiClientSecure httpClient;

 if (!httpClient.connect(host, 8080)) {
                Serial.println("connection failed");
                return;
                }
 if (httpClient.verify(fprint,host)) {
              Serial.println("certificate match");
  } else {
            Serial.println("certificate doesn't match");
            return;
            }

It returns with "certificate match" . However, continuing with

result = ESPhttpUpdate.update(host,8080,URI, String(""),String(fprint));

returns with the dreaded "connection refused". I tried

HTTPClient httpClient;
httpClient.begin(host,8080,URI,fprint)

and that also worked ok. I was able to see the URI in the servers log-file. But any attempt to do

ESPhttpUpdate.update(..)

fails with connection refused without log-file entry on the server. So - ESPhttpUpdate.update(..) doesn't even come to the point where it actually requests anything.

HOWEVER: Changing the URI to something that ends in ".bin"

EC:11 PID:21297 OTA Update request: [/ota/test/works.bin]

allowed the update-process to proceed (server log entry above). So it seems the "Connection refused" error is .. emm .. bogus. Has nothing to do with a connection, but the filename(URL, URI) needs to be correct.

Hope that helps.

[CRCinAU]

So as this is still open, does anyone have an example of using the BearSSL implementation with httpUpdate?

I'm happy to start tinkering here, but can't quite figure out how to make it all fit together...