First UDP packet always lost

[AndreKR]

Hardware: ESP-12F
Core Version: 2.3.0

I'm sending four UDP packets, with payload "aaa", "bbb", "ccc" and "ddd".

The first of those packets never appears on the receiving machine. It happens absolutely every time, the first packet is always missing.

void setup() 
{ 
  Serial.begin(115200);
  Serial.println("Start");
  Serial.println("Re-setting Wifi credentials");
  WiFi.begin(ssid, password); 

  while(WiFi.status() != WL_CONNECTED)
    delay(50);
  
  Serial.println("My IP address:");
  Serial.println(WiFi.localIP());
  Serial.println("Sending");
  Serial.println(Udp.beginPacket("192.168.10.2", 7827));
  Serial.println(Udp.print("aaa"));
  Serial.println(Udp.endPacket());
  Serial.println(Udp.beginPacket("192.168.10.2", 7827));
  Serial.println(Udp.print("bbb"));
  Serial.println(Udp.endPacket());
  Serial.println(Udp.beginPacket("192.168.10.2", 7827));
  Serial.println(Udp.print("ccc"));
  Serial.println(Udp.endPacket());
  Serial.println(Udp.beginPacket("192.168.10.2", 7827));
  Serial.println(Udp.print("ddd"));
  Serial.println(Udp.endPacket());
  delay(100); // TODO: find out when sending is finished
  goToSleep();
}

Serial output:

Start
Re-setting Wifi credentials
My IP address:
192.168.10.196
Sending
1
3
1
1
3
1
1
3
1
1
3
1
Going to sleep

TCP dump on the receiving machine:

21:27:29.474470 IP 192.168.10.196.4097 > 192.168.10.2.7827: UDP, length 3
        0x0000:  4500 001f 0003 0000 8011 a4b4 c0a8 0ac4  E...............
        0x0010:  c0a8 0a02 1001 1e93 000b 76ca 6262 6200  ..........v.bbb.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
21:27:29.476750 IP 192.168.10.196.4097 > 192.168.10.2.7827: UDP, length 3
        0x0000:  4500 001f 0004 0000 8011 a4b3 c0a8 0ac4  E...............
        0x0010:  c0a8 0a02 1001 1e93 000b 74c9 6363 6300  ..........t.ccc.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
21:27:29.477491 IP 192.168.10.196.4097 > 192.168.10.2.7827: UDP, length 3
        0x0000:  4500 001f 0005 0000 8011 a4b2 c0a8 0ac4  E...............
        0x0010:  c0a8 0a02 1001 1e93 000b 72c8 6464 6400  ..........r.ddd.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............

[luffykesh]

I also have faced this issue... First, I thought there was some issue with my code, but there is an issue with the library or the wifi chip..

[hridder]

As a test, try putting a 1 sec delay between UDP sends. It's possible that the library buffers (or drops) the packet until the ARP request/response completes in order to find the receivers MAC address. Note that by definition, UDP is unreliable and packet loss should be expected. It might be surprising but the library might technically be correct if even if it dropped packets until it gets an ARP reply...

[AndreKR]

It seems that even a delay(1) after endPacket() can fix it.

[hridder]

Which suggests it's the delay required for the ARP response. If so, then there's probably a race between getting the response and your sketch making the next call (either beginPacket() or endPacket(), you'd have to look at the code). Note that if the receiving system is slow, or the network is busy you might find a 1 ms delay isn't quite enough... and you'll lose a packet again. TCP is there to avoid these kind of issues.

In any case, it's probably not a bug.

[liebman]

In order to get more accurate timestamps with NTP I have taken to "ping" the IP address I'm sending to first to force the arp request.

[AndreKR]

Unfortunately the Espressif SDK is not open source, so we can't just look at what's going on. I would have expected endPacket() to block until the packet is sent. Can it be related to having to yield the CPU to the firmware?

[suculent]

Why don't just use esp-open-sdk and do what you want to? Odesláno z iPhonu 6. 9. 2017 v 22:54, AndreKR <notifications@github.com>:

…

Unfortunate the Espressif SDK is not open source, so we can't just look at what's going on. I would have expected endPacket() to block until the packet is sent. Can it be related to having to yield the CPU to the firmware? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[herrold]

@AndreKR said:
--- Unfortunately the Espressif SDK is not open source

I thought that their SDK, which seems to be based on the 15 year old 'lwIP' stack
https://savannah.nongnu.org/projects/lwip/

was here:
https://github.com/espressif/ESP8266_RTOS_SDK

While it purports to have a 'this hardware only' MIT License, I don't really see how that would not be a derivative work from the plain, non-restricted lwIP one

[AndreKR]

@herrold
The ARP resolution happens in this netif->output() function and if I see this correctly that comes from libnet80211.a which is binary and no one knows where it comes from.

@suculent It's unclear to me how to use it with Arduino.

[devyte]

@AndreKR is this still valid with latest git? with latest git and lwip2? (you can select lwip2 from the Arduino IDE menu).

[AndreKR]

How to use the git version?
I deleted
%LOCALAPPDATA%\Arduino15\packages\esp8266\hardware\esp8266\2.3.0
and instead downloaded the master branch and put it in %LOCALAPPDATA%\Arduino15\packages\esp8266\hardware\esp8266com\esp8266
as described in the README. Then I cd'ed into tools and ran python get.py which completed without errors.
When I now start the Arduino IDE, there are no ESP8266 board available in Tools -> Boards.

[igrr]

When installing git version, it should not go into Appdata, instead place it into 'hardware' directory either under Arduino IDE directory or your sketchbook direcotry.

…

On Fri, Nov 17, 2017, 22:58 AndreKR ***@***.***> wrote: How to use the git version? I deleted %LOCALAPPDATA%\Arduino15\packages\esp8266\hardware\esp8266\2.3.0 and instead downloaded the master branch and put it in %LOCALAPPDATA%\Arduino15\packages\esp8266\hardware\esp8266com\esp8266 as described in the README. Then I cd'ed into tools and ran python get.py which completed without errors. When I now start the Arduino IDE, there are no ESP8266 board available in Tools -> Boards. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3481 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEJcesIR5jCWY90smQzSjy2VDw89mkd_ks5s3Z8XgaJpZM4Oo0pH> .

[AndreKR]

Yes, this issue still happens with the current git master and "IwIP Variant" set to "v2", which was the default.

[d-a-v]

On current master, I don't have the issue with lwip2.
endPacket() returns 0 if the packet is not sent. Logs below.

However it indeed does not work with lwip1.4.
Lots of packets are lost and endPacket() returns always 1. Only the two or three last packets are received. Like if the new packets would overwrite the still not sent previous ones.

Here is my complete sketch.
Can you double check with lwip2, and, if the problem persists, post your complete sketch ?

#include <ESP8266WiFi.h>
#include <WiFiUdp.h>

WiFiUDP udp;

void setup()
{
  Serial.begin(115200);
  Serial.println("Start");
  WiFi.begin("my", "wifi");

  while (WiFi.status() != WL_CONNECTED)
    delay(50);

  udp.begin(1234);

  Serial.println("My IP address:");
  Serial.println(WiFi.localIP());
  Serial.println("Sending");

  char* data = "xxx";

  for (int i = 0; i < 10; i++)
  {
    data[0] = 'a' + i;
    Serial.print(data);
    Serial.print(" ");
    Serial.print(udp.beginPacket("192.168.1.8", 7827));
    Serial.print(" ");
    Serial.print(udp.print(data));
    Serial.print(" ");
    Serial.print(udp.endPacket());
    Serial.println(" ");
  }
  //delay(100); // TODO: find out when sending is finished
  //goToSleep();
}

void loop() {
  // put your main code here, to run repeatedly:

}

Start
My IP address:
192.168.1.239
Sending
axx 1 3 1 
bxx 1 3 1 
cxx 1 3 1 
dxx 1 3 1 
exx 1 3 1 
fxx 1 3 1 
gxx 1 3 1 
hxx 1 3 1 
ixx 1 3 0 
jxx 1 3 0 

15:58:16.882875 IP 192.168.1.239.1234 > 192.168.1.8.7827: UDP, length 3
        0x0000:  4500 001f 0007 0000 ff11 377f c0a8 01ef  E.........7.....
        0x0010:  c0a8 0108 04d2 1e93 000b 7eb2 6178 7800  ..........~.axx.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
15:58:16.883610 IP 192.168.1.239.1234 > 192.168.1.8.7827: UDP, length 3
        0x0000:  4500 001f 0008 0000 ff11 377e c0a8 01ef  E.........7~....
        0x0010:  c0a8 0108 04d2 1e93 000b 7db2 6278 7800  ..........}.bxx.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
15:58:16.884651 IP 192.168.1.239.1234 > 192.168.1.8.7827: UDP, length 3
        0x0000:  4500 001f 0009 0000 ff11 377d c0a8 01ef  E.........7}....
        0x0010:  c0a8 0108 04d2 1e93 000b 7cb2 6378 7800  ..........|.cxx.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
15:58:16.889014 IP 192.168.1.239.1234 > 192.168.1.8.7827: UDP, length 3
        0x0000:  4500 001f 000a 0000 ff11 377c c0a8 01ef  E.........7|....
        0x0010:  c0a8 0108 04d2 1e93 000b 7bb2 6478 7800  ..........{.dxx.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
15:58:16.889689 IP 192.168.1.239.1234 > 192.168.1.8.7827: UDP, length 3
        0x0000:  4500 001f 000b 0000 ff11 377b c0a8 01ef  E.........7{....
        0x0010:  c0a8 0108 04d2 1e93 000b 7ab2 6578 7800  ..........z.exx.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
15:58:16.897363 IP 192.168.1.239.1234 > 192.168.1.8.7827: UDP, length 3
        0x0000:  4500 001f 000c 0000 ff11 377a c0a8 01ef  E.........7z....
        0x0010:  c0a8 0108 04d2 1e93 000b 79b2 6678 7800  ..........y.fxx.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
15:58:16.903304 IP 192.168.1.239.1234 > 192.168.1.8.7827: UDP, length 3
        0x0000:  4500 001f 000d 0000 ff11 3779 c0a8 01ef  E.........7y....
        0x0010:  c0a8 0108 04d2 1e93 000b 78b2 6778 7800  ..........x.gxx.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............
15:58:16.905036 IP 192.168.1.239.1234 > 192.168.1.8.7827: UDP, length 3
        0x0000:  4500 001f 000e 0000 ff11 3778 c0a8 01ef  E.........7x....
        0x0010:  c0a8 0108 04d2 1e93 000b 77b2 6878 7800  ..........w.hxx.
        0x0020:  0000 0000 0000 0000 0000 0000 0000       ..............