ArduinoOta with esp8266 Network congestion

[ejbp]

Hardware: Olimex ESP8266-dev
ArduinoOta: v5.11.1

I'm experience network problems during OTA updating process. Basically when I'm uploading the sketch to my esp8266 the local network (on all devices) immediately goes from pinging 2ms/3ms to 300ms/1000ms with packets loss.

PING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=593.825 ms
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=853.687 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=458.956 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=400.839 ms
64 bytes from 192.168.1.1: icmp_seq=4 ttl=64 time=596.334 ms
Request timeout for icmp_seq 5
64 bytes from 192.168.1.1: icmp_seq=5 ttl=64 time=1008.485 ms
[..]
64 bytes from 192.168.1.1: icmp_seq=22 ttl=64 time=1087.114 ms
64 bytes from 192.168.1.1: icmp_seq=23 ttl=64 time=205.291 ms
64 bytes from 192.168.1.1: icmp_seq=24 ttl=64 time=503.261 ms
64 bytes from 192.168.1.1: icmp_seq=25 ttl=64 time=1036.019 ms
64 bytes from 192.168.1.1: icmp_seq=26 ttl=64 time=34.023 ms
64 bytes from 192.168.1.1: icmp_seq=27 ttl=64 time=22.274 ms
64 bytes from 192.168.1.1: icmp_seq=28 ttl=64 time=191.950 ms
64 bytes from 192.168.1.1: icmp_seq=29 ttl=64 time=229.213 ms
64 bytes from 192.168.1.1: icmp_seq=30 ttl=64 time=525.463 ms
[...After_Done_Uploading...] 
64 bytes from 192.168.1.1: icmp_seq=31 ttl=64 time=103.161 ms
64 bytes from 192.168.1.1: icmp_seq=32 ttl=64 time=171.275 ms
64 bytes from 192.168.1.1: icmp_seq=33 ttl=64 time=1.967 ms
64 bytes from 192.168.1.1: icmp_seq=34 ttl=64 time=2.687 ms
64 bytes from 192.168.1.1: icmp_seq=35 ttl=64 time=2.675 ms
64 bytes from 192.168.1.1: icmp_seq=36 ttl=64 time=1.637 ms
64 bytes from 192.168.1.1: icmp_seq=37 ttl=64 time=6.304 ms
64 bytes from 192.168.1.1: icmp_seq=38 ttl=64 time=3.494 ms
64 bytes from 192.168.1.1: icmp_seq=39 ttl=64 time=2.415 ms

Settings in IDE

Module: Olimex Mod-Wifi-ESP8266(-dev)
[..]core_esp8266_esp8266_modwifi_CpuFrequency_80,UploadSpeed_460800_2f159fcebb45fcbf58350f9e858330c3.a
Sketch uses 285735 bytes (27%) of program storage space. Maximum is 1044464 bytes.
Global variables use 35632 bytes (43%) of dynamic memory, leaving 46288 bytes for local variables. Maximum is 81920 bytes.
Uploading via ArduinoOTA

Sketch

main.ino

void loop() {
 [...]
  
  if (otaHandler->isOtaActive()) { //this returns true
    otaHandler->cycle();

    if (otaHandler->isOtaUpgrading()) { //this returns true
      return;
    }
    
  }

  [...]
}

OtaHandler.cpp

#include "OtaHandler.h"

static bool isActive = false;
static bool isUpgrading = false;
  
OtaHandler::OtaHandler() {

}

void OtaHandler::activateOta() {
    if (isActive || WiFi.status() != WL_CONNECTED) return;
    isActive = true;
    
    String hostname_str = "Biti " + Utils::getMacAddress();
    char ota_hostname[hostname_str.length()];
    hostname_str.toCharArray(ota_hostname, hostname_str.length()); 
    
    ArduinoOTA.setHostname( ota_hostname);
    ArduinoOTA.setPassword("xxxx");

    ArduinoOTA.onStart([]() {
      setState(true, true);
      /*String type;

      if (ArduinoOTA.getCommand() == U_FLASH)
        type = "sketch";
      else // U_SPIFFS
        type = "filesystem";
  
      // NOTE: if updating SPIFFS this would be the place to unmount SPIFFS using SPIFFS.end()
      Serial.println("Start updating " + type);*/
      Serial.println("Start updating via OTA");
    });

    ArduinoOTA.onEnd([]() {
      Serial.println("\nEnd");
      //setState(true, false);
    });

    ArduinoOTA.onProgress([](unsigned int progress, unsigned int total) {
      Serial.printf("Progress: %u%%\r", (progress / (total / 100)));
    });

    ArduinoOTA.onError([](ota_error_t error) {
      Serial.printf("Error[%u]: ", error);
      setState(true, false);
      
      if (error == OTA_AUTH_ERROR) Serial.println("Auth Failed");
      else if (error == OTA_BEGIN_ERROR) Serial.println("Begin Failed");
      else if (error == OTA_CONNECT_ERROR) Serial.println("Connect Failed");
      else if (error == OTA_RECEIVE_ERROR) Serial.println("Receive Failed");
      else if (error == OTA_END_ERROR) Serial.println("End Failed");
    });

    ArduinoOTA.begin();
    Serial.print("Ota Ready. IP address: ");
    Serial.println(WiFi.localIP());
}

void OtaHandler::desactivateOta() {
    if ( !isOtaActive() ) return;
    isActive = false;
}

void OtaHandler::cycle() {
    if ( isOtaActive() || isOtaUpgrading() ) {
        ArduinoOTA.handle();
    }
}

void OtaHandler::setState(bool _isActive, bool _isUpgrading) {
    
    isActive = _isActive;
    isUpgrading = _isUpgrading;
    
}

Note:I always receive Upload Complete message but sometimes/randomly the firmware is not updated.

Thanks

[igrr]

While performing OTA update, ESP8266 needs to erase flash sectors before writing new data to them. Erasing flash takes a lot of time, and during that time LwIP stack can not handle any incoming packets, because flash cache is disabled.

Can you explain the use case a bit, why should the ESP8266 remain responsive to pings during OTA update?

[ejbp]

The ping is not for the esp8266 but for the router. What I'm describing is that it congests all the network. So, during the OTA process, all devices(computers/mobiles/etc) connected to that router, starts to loose packets, that's why the title is network congestion. I would understand if the esp8266 stops responding to pings, but I don't understand why the OTA process affects the entire network (at the least the ones connected through wifi, I didn't tested with wired connections).

I'm doing OTA via Arduino IDE 1.8.4.

[igrr]

Okay, got it. In this case, suggest performing capture of the WiFi network activity using Wireshark. That would help determine if the ESP8266 is generating some extra traffic while doing the OTA. FWIW, i don't see such behaviour on my local network, so this may be dependent on the wireless router. In any case, wireshark capture will likely provide extra information.

[ejbp]

That's a good idea. I'll try that and give feedback.

[rjlexx]

Hello, ejbp. Did you try to capture WiFi network activity? It looks like I have a similar issue after upgrading to the latest git version. Sometimes OTA update or SPIFFS OTA upload fails with random ERROR
Uploading...........................................................................................................................................................................................
19:48:37 [ERROR]: K
SPIFFS Upload failed!

Iggr, on the nightly build v2.4.0 I didn't have the issue. But, it was another issue with long downloading static data from SPIFFS.

[ejbp]

Sorry, I didn't try it. I've changed ESP to update itself by connecting to a http server, download the image and self-install. Though, if you figure out the answer, please post it here. It will be very helpful.