WiFi dropouts unless using WiFi.setSleepMode(WIFI_NONE_SLEEP)

[CRCinAU]

I've noticed lately that multiple devices I have have started dropping off WiFi and reconnecting (causing MQTT dropouts etc) unless I specify WiFi.setSleepMode(WIFI_NONE_SLEEP) when setting up the WiFi adapter.

ESP.getFullVersion() output:

SDK:3.0.0-dev(c0f7b44)/Core:2.5.0=20500000/lwIP:STABLE-2_1_2_RELEASE/glue:1.1/BearSSL:6778687

WiFi setup code in setup():

mConnectHandler = WiFi.onStationModeConnected(onConnected);
mDisConnectHandler = WiFi.onStationModeDisconnected(onDisconnect);
mGotIpHandler = WiFi.onStationModeGotIP(onGotIP);

WiFi.disconnect() ;
WiFi.persistent(false);
WiFi.setSleepMode(WIFI_NONE_SLEEP);
WiFi.mode(WIFI_STA);            // Client mode
WiFi.setOutputPower(17);        // 10dBm == 10mW, 14dBm = 25mW, 17dBm = 50mW, 20dBm = 100mW
WiFi.begin(ssid, password);     // Start WiFi.

And the functions:

void onConnected(const WiFiEventStationModeConnected& event){
update_status += millis();
update_status += ": Connected to AP.";
update_status += "\n";
}
void onDisconnect(const WiFiEventStationModeDisconnected& event){
update_status += millis();
update_status += ": Station disconnected";
update_status += "\n";
client.disconnect();
}
void onGotIP(const WiFiEventStationModeGotIP& event){
update_status += millis();
update_status += ": Station connected, IP: ";
update_status += WiFi.localIP().toString();
update_status += "\n";
timer.setTimeout(1000, checkForUpdate);
}

update_status is a global string that is included on the web page interface.

If I comment out the setSleepMode line, I get regular wifi disconnections.

[CRCinAU]

Issue moved across from platformio/platform-espressif8266#135

[d-a-v]

Can you try that and report back wether it is sufficient without WiFi.setSleepMode(WIFI_NONE_SLEEP) (regarding power consumption) ?
#2330 (comment)

[CRCinAU]

I've implemented this as a SimpleTimer via:

void GratuitousARPTask() {
        netif *n = netif_list;

        while (n) {
                etharp_gratuitous(n);
                n = n->next;
        }
}

SimpleTimer then triggers every ~5000ms for that task.

I can see the following on the network via tcpdump:

03:21:22.728282 ARP, Request who-has 10.1.1.11 tell 10.1.1.11, length 46
03:21:27.704425 ARP, Request who-has 10.1.1.11 tell 10.1.1.11, length 46
03:21:32.699405 ARP, Request who-has 10.1.1.11 tell 10.1.1.11, length 46
03:21:37.726173 ARP, Request who-has 10.1.1.11 tell 10.1.1.11, length 46
03:21:42.708973 ARP, Request who-has 10.1.1.11 tell 10.1.1.11, length 46
03:21:47.688705 ARP, Request who-has 10.1.1.11 tell 10.1.1.11, length 46
03:21:52.726033 ARP, Request who-has 10.1.1.11 tell 10.1.1.11, length 46

I've also disabled the WiFi.setSleepMode(WIFI_NONE_SLEEP);, so we'll see what happens.

[CRCinAU]

Setting WIFI_LIGHT_SLEEP and sending GARP's - and it doesn't seem to fix the issue. I don't have any of the ESP8266 units with over an hour uptime according to the AP.

I even tried setting one to 802.11g mode, no real difference.

[d-a-v]

Have you tried with latest git which uses a more stable espressif firmware ?

[CRCinAU]

Embarrassingly, I can't quite remember what to feed into PlatformIO to try this.

Currently, I have the following in platformio.ini:

platform = espressif8266
;platform = https://github.com/platformio/platform-espressif8266.git
;platform = https://github.com/platformio/platform-espressif8266.git#feature/stage

I'm pretty sure none of these are directly from this repo. I'm pretty sure I've done this before - but I can't find my notes that mention it. Any hints on being able to get this to reflect here instead?

[Jason2866]

This entry use latest ESP8266 dev version: platform = https://github.com/platformio/platform-espressif8266.git#feature/stage

[CRCinAU]

This entry use latest ESP8266 dev version: platform = https://github.com/platformio/platform-espressif8266.git#feature/stage

That doesn't map up with whats in this repo though....

From https://github.com/platformio/platform-espressif8266/tree/feature/stage (in your browser):
Latest commit 645d2d9 14 days ago

The commit list here has:
Latest commit cdb5495 2 days ago

I'm not sure the two compare...

[Jason2866]

You have to force platformio to update!

[CRCinAU]

Ah! That's what I was missing....

Platform Espressif 8266 (Stage)
--------
Updating espressif8266                   @ 645d2d9        [Up-to-date]
Updating framework-arduinoespressif8266  @ cdb5495        [Up-to-date]
Updating tool-esptool                    @ 1.413.0        [Up-to-date]
Updating toolchain-xtensa                @ 1.40802.0      [Up-to-date]

Let me start testing again.

[CRCinAU]

Woah, this seems to lead to hard hangs if I set WiFi.setSleepMode(WIFI_LIGHT_SLEEP);

I hit the reset button to bring it back to life after each of the disconnection timeouts below. Setting WiFi.setSleepMode(WIFI_NONE_SLEEP); seems to restore functionality with no hangs.

Version string is now:

SDK:2.2.1(cfd48f3)/Core:2.6.0-dev=20600000/lwIP:STABLE-2_1_2_RELEASE/glue:1.1-7-g82abda3/BearSSL:a143020

MQTT logs for timing:

Apr 28 23:11:33 mosquitto[538]: 1556457093: New connection from 10.1.1.11 on port 1883.
Apr 28 23:11:33 mosquitto[538]: 1556457093: New client connected from 10.1.1.11 as 84:F3:EB:10:75:2C (c1, k15).                                                                                                                
Apr 28 23:12:08 mosquitto[538]: 1556457128: Client 84:F3:EB:10:75:2C has exceeded timeout, disconnecting.                                                                                                                      
Apr 28 23:12:08 mosquitto[538]: 1556457128: Socket error on client 84:F3:EB:10:75:2C, disconnecting.
Apr 28 23:13:28 mosquitto[538]: 1556457208: New connection from 10.1.1.11 on port 1883.
Apr 28 23:13:28 mosquitto[538]: 1556457208: New client connected from 10.1.1.11 as 84:F3:EB:10:75:2C (c1, k15).                                                                                                                
Apr 28 23:13:52 mosquitto[538]: 1556457232: Client 84:F3:EB:10:75:2C has exceeded timeout, disconnecting.                                                                                                                      
Apr 28 23:13:52 mosquitto[538]: 1556457232: Socket error on client 84:F3:EB:10:75:2C, disconnecting.

[TD-er]

Make sure you do a clean build and build again after updating the core library, when using PlatformIO.
Most of the time it works just fine, but I know from experience that it may cost a lot of hours debugging to find not reproducible error reports, which seem to be fixed after a clean build.

Apart from that, I've seen similar issues.
Gratuitous ARP does seem to help a bit, but it isn't the fix for all.

What I'm seeing is that the node does reduce power somehow (you can measure its power consumption) and then starts to miss packets until it receives some ping requests (or other repeated network requests)
To answer these requests, the current usage increases for a while and all requests are answered well.

Even when setting the wifi to none sleep, the power consumption may drop after a while and then it starts to miss packets.
Strange thing is, it will never miss ICMP packets. It may take up-to a few-100 msec to reply the first one, but all ping packets will get a reply.

Now the really strange part....
Some builds run fine for weeks and some can hardly stay up for a few hours to days.
So it somewhat looks to me like some uninitialized variable or some array being addressed out of bound, or an alignment issue. Behavior of these can all be different by some changes in totally unrelated code.

[lp422003]

When do you call the function Gratuitous ARP, during the loop correct?

[arihantdaga]

@TD-er How can we use Gratuitous ARP. Is there any temporay solution to this problem. I went through few threads related to this on this repo as well as EspEasy's repo, i saw that you have worked on it a lot. Did you find some temporary or permanent solution to this problem ?

[TD-er]

I do send Gratuitous ARP packets when just connected and when I encounter failed connect attempts (also DNS lookup)
And also periodically I do send them, when it is enabled in the settings.

Another approach can be to send ping packets to the node, since ICMP packets are somehow treated differently. Not sure if it is only on the ESP, or also in the AP what makes it act different.
You can see its power consumption remain constant (high) when it is receiving pings, but also its reaction time improves and that must be something in the AP I guess.

I am not (yet) using it on the ESP32 and you can clearly see a difference in how well it does react on web requests or the UDP packets sent among ESPeasy nodes.
The ESP32 does only receive a handful of them and thus lists only a fraction of the ESPeasy nodes in the network. Some listed nodes have a "last seen" timestamp of 5 - 10 minutes, while each node does send its presence every minute.