OTA network ports show in all 1.8.X versions but not in any 2.X versions of IDE

[HomeSpan]

Describe the problem

I am using the ArduinoOTA library that is included with Arduino-ESP32 (version 2.0.2) and under Arduino 1.8.19 every works fine. I can see all of my ESP2 devices under the Port menu in 1.8.19. I can also see the MDNS broadcasts are working. However, none of the devices show up under Arduino IDE 2.0.0 RC 6 (or any previous versions). Based on other posted issues it appears that there was some problem with showing device names, but nothing about the lack of any devices showing up at all. Are there other MDNS items that must be broadcast that are required by 2.0.0 that were not required by 1.8.19? Or perhaps a preference setting in IDE 2.0.0 that enables network ports to be shows?

To reproduce

Compile and load ArduinoOTA->BasicOTA Example from Arduino-ESP32. After running, board shows up as network port under IDE 1.8.19, but does show up an network port under 2.0.0 RC6.

Expected behavior

Expected board to show up under network port within 2.0.0 RC 6.

Arduino IDE version

2/0/0

Operating system

macOS

Operating system version

12.3.1

Additional context

No response

Issue checklist

• I searched for previous reports in the issue tracker
• I verified the problem still occurs when using the latest nightly build
• My report contains all necessary details

[datort]

I'm experiencing the same issue. Arduino 1.8.x works like a charm and lists my NodeMCU ESP boards for OTA update right after the start, but 2.0.0-rc6 doesn't show any board at all. Running the Arduino 2.0.0-rc6 IDE on an Intel Mac.

[per1234]

Thanks @HomeSpan and @datort.

I think the best way to proceed with the investigation is to eliminate any unnecessary factors from the equation. So I'm going to ask you to to an experiment and report your results.

The network port discovery is done using a tool named mdns-discovery. If the problem still occurs when you use that tool directly, then we can eliminate the Arduino IDE and Arduino CLI from consideration as being related to the problem.

❗ NOTE: The following instructions will not solve the problem. They are only intended to gather additional information.

I'll provide the instructions here:

During all this, keep an eye out for anything that doesn't match the expected behavior as described at each step.

1. Open your Applications folder.
2. Open Utilities.
3. Double-click on "Terminal".
4. Run this command:

   ls ~/Library/Arduino15/packages/builtin/tools/mdns-discovery
   

5. You will now see a list of the versions of mdns-discovery which are installed on your computer. Take note of the highest version on the list (recent versions of Arduino IDE 2.x always use the newest one)
   For example, if you see output like this:

   0.9.2   1.0.2   1.0.5
   

   then the newest version is "1.0.5".
6. Run the following command:

   ~/Library/Arduino15/packages/builtin/tools/mdns-discovery/<newest version>/mdns-discovery
   

   (where <newest version> is the version number you found at step 3)
   mdns-discovery will now start. There won't be any obvious sign of this other than that there is no longer a command prompt at the cursor in the terminal.
7. Type the following command:

   HELLO 1 "arduino-cli 0.21.0"
   

8. Press the Enter key.
   You should now see a response printed exactly like this:

   {
     "eventType": "hello",
     "protocolVersion": 1,
     "message": "OK"
   }

9. Unplug your Arduino board from your computer if you have it plugged in.
10. Type the following command:

    START_SYNC
    

11. Press the Enter key.
    You should now see a response printed exactly like this:

    {
      "eventType": "start_sync",
      "message": "OK"
    }

    You might also see some additional objects in the output depending on which network ports are available on your computer.
12. Plug your Arduino board into your computer
    You should eventually see a response printed that looks something like this:

    {
      "eventType": "add",
      "port": {
        "address": "192.168.254.127",
        "label": "esp32-b4e62dbf693d at 192.168.254.127",
        "protocol": "network",
        "protocolLabel": "Network Port",
        "properties": {
          ".": "node32s",
          "auth_upload": "no",
          "board": "node32s",
          "hostname": "esp32-b4e62dbf693d.local.",
          "port": "3232",
          "ssh_upload": "no",
          "tcp_check": "no"
        }
      }
    }

    ⓘ This is only an example of what you might see. The output will be different depending on the board you connected.
13. Unplug your Arduino board from your computer
    You should eventually see a response printed that looks something like this:

    {
      "eventType": "remove",
      "port": {
        "address": "192.168.254.127",
        "label": "esp32-b4e62dbf693d at 192.168.254.127",
        "protocol": "network",
        "protocolLabel": "Network Port",
        "properties": {
          ".": "node32s",
          "auth_upload": "no",
          "board": "node32s",
          "hostname": "esp32-b4e62dbf693d.local.",
          "port": "3232",
          "ssh_upload": "no",
          "tcp_check": "no"
        }
      }
    }

    ⓘ This is only an example of what you might see. The port data should be the same as the "add" event you saw when you plugged the board in.

You should continue to see the same results if you repeat steps (10) and (11).

Once you are done with your experiments with the "mdns-discovery" tool, follow these instructions to exit:

1. Type the following command:

   STOP
   

2. Press the Enter key.
   You should now see a response printed exactly like this:

   {
     "eventType": "stop",
     "message": "OK"
   }

3. Type the following command:

   QUIT
   

4. Press the Enter key.
   You should now see a response printed exactly like this:

   {
     "eventType": "quit",
     "message": "OK"
   }

   You should now be back at the shell command line.

[datort]

Hey @per1234,

thanks for your support. I tried this but failed at 10. with this after the "START SYNC":

{
  "eventType": "start_sync",
  "message": "mdns lookup error: write udp6 [::]:52509-\u003e[ff02::fb]:5353: sendto: no route to host",
  "error": true
}
{
  "eventType": "start_sync",
  "message": "mdns lookup error: write udp6 [::]:58447-\u003e[ff02::fb]:5353: sendto: no route to host",
  "error": true
}
{
  "eventType": "start_sync",
  "message": "mdns lookup error: write udp6 [::]:54128-\u003e[ff02::fb]:5353: sendto: no route to host",
  "error": true
}
{
  "eventType": "start_sync",
  "message": "mdns lookup error: write udp6 [::]:60031-\u003e[ff02::fb]:5353: sendto: no route to host",
  "error": true
}
....

This loops all over again. Any suggestions? Shall I continue?

Kind Regards

PS: It's v1.05 for the discovery tool

[per1234]

Thanks @datort! That is very useful. I can see that you are experiencing the issue tracked here: #24

#24 is almost entirely lacking in any useful detail, but I can see from a reference to it in another issue that it does indeed produce errors that follow the same format: arduino/arduino-cli#1543 (comment)

So we know that this is the result of a deficiency in the mdns-discovery tool for you, and not anything specific to the Arduino IDE 2.x code base. I'll wait for the output from @HomeSpan to see whether their port discovery failure is also caused by the same.

Sadly, I am not aware of any workaround for the issue other than using Arduino IDE 1.x until there is a real fix.

[HomeSpan]

I just tried and have exactly the same issue - the v1.0.5 version loops with the same error message when I issue START_SYNC.

However, I also have v0.9.2 on my system and when I ran that version it did not loop in step 10 - it simply responded with start_sync being OK (in JSON format). To try this version with the Arduino IDE I deleted the 1.0.5 directory, thinking that the IDE would automatically use version 0.9.2, but it appears that the Arduino 2.0 IDE automatically installs version 1.0.5 during startup if that version is not found.

To get around this, I renamed the file mdns-discovery in the 1.0.5 folder to something else, and I copied the mdns-discovery executable file from the 0.9.2 folder into the 1.0.5 folder. I then restarted the Arduino 2.0 IDE and...it worked! All my devices that broadcast OTA now correctly show up in the Arduino 2.0 IDE as network ports.

As noted in some other threads, the only info provided is the IP address of each board, not the hostname (which is provided in Arduino 1.8.19). I suppose this is the issue that was fixed in v1.0.5, but something else must have broke that now causes it to completely fail, whereas v0.9.2 still works (though with the limitations of only showing the IP address).

Can a bug-report be created for v1.0.5?