Recover from interrupted discovery/monitor installation

[per1234]

Describe the request

Reinstall pluggable discovery and monitor tools of the builtin package if the tool executable is not found.

Describe the current behavior

Arduino CLI automatically installs pluggable discovery and pluggable monitor tools of the builtin package when missing or updatable. These tools allow Arduino CLI to find the "ports" used for communication with Arduino boards.

It can happen that the installation process is interrupted or interfered with.

🙁 If a problem during the installation results in the creation of the tool's release installation folder without the tool executable, Arduino CLI can no longer do discovery. It will not automatically recover from this condition and the user must manually delete the incomplete installation in order to restore Arduino CLI to functionality.

To reproduce

$ arduino-cli version
arduino-cli.exe  Version: git-snapshot Commit: 2c2a5cc6 Date: 2023-04-22T13:21:59Z

$ ls "C:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\builtin\\tools\\serial-discovery"
1.4.0/

$ ls "C:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\builtin\\tools\\serial-discovery\\1.4.0"
LICENSE.txt  serial-discovery.exe*

$ rm "C:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\builtin\\tools\\serial-discovery\1.4.0\\serial-discovery.exe"

$ arduino-cli board list
Error starting discovery: discovery builtin:serial-discovery process not started: exec: "C:\\Users\\per\\AppData\\Local\\Arduino15\\packages\\builtin\\tools\\serial-discovery\\1.4.0\\serial-discovery": file does not exist
No boards found.

Arduino CLI version

2c2a5cc

Operating system

Windows, Linux

Operating system version

Windows 11, Ubuntu 22.04

Additional context

Originally reported at https://forum.arduino.cc/t/ide-2-0-lost-all-com-ports/960650

---

This situation is more problematic for Arduino IDE users, since the error message about the missing tool is not visible to them and they only notice the symptom of the IDE not seeing their ports.

---

It might seem unlikely for this interrupted installation to occur under real world conditions, but the equivalent issue with Boards Manager producing incomplete tools installations has been reported by users periodically ever since the introduction of Boards Manager, at sufficient volume for Arduino Support to consider it worth a Help Center article:

https://support.arduino.cc/hc/en-us/articles/360018444739-Error-file-does-not-exist-no-such-file-or-directory-system-cannot-find-the-file-specified

[per1234]

This now results in a cryptic panic that gives no clue about the cause to mere mortals:

$ arduino-cli version
arduino-cli  Version: 0.25.1-rc1 Commit: 436f0bb9 Date: 2022-07-23T14:28:16Z

$ rm ~/.arduino15/packages/builtin/tools/serial-discovery/1.3.2/*

$ ./arduino-cli board list
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4a51d9]

goroutine 35 [running]:
os.(*Process).signal(0xc000174770, {0x1034aa0, 0x15fd848})
	/usr/local/go/src/os/exec_unix.go:64 +0x39
os.(*Process).Signal(...)
	/usr/local/go/src/os/exec.go:138
os.(*Process).kill(...)
	/usr/local/go/src/os/exec_posix.go:68
os.(*Process).Kill(...)
	/usr/local/go/src/os/exec.go:123
github.com/arduino/arduino-cli/executils.(*Process).Kill(...)
	/home/per/Documents/deleteme/arduino-cli/executils/process.go:120
github.com/arduino/arduino-cli/arduino/discovery.(*PluggableDiscovery).killProcess(0xc0000b5480)
	/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discovery.go:268 +0xc9
github.com/arduino/arduino-cli/arduino/discovery.(*PluggableDiscovery).Quit(0xc0000b5480)
	/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discovery.go:381 +0xde
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).remove(0xc000070a20, {0xc000017dfe, 0x18})
	/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:82 +0xba
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).RunAll.func1(0xc0000b5480)
	/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:131 +0x71
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).parallelize.func1(0x1)
	/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:101 +0x68
created by github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).parallelize
	/home/per/Documents/deleteme/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:99 +0x1c5

The demo was done using Linux, but the problem also occurs on Windows.

[cmaglie]

Is this happening when there is only the directory without the executable?

[per1234]

That is correct.

$ arduino-cli version
arduino-cli.exe  Version: 0.25.1-rc1 Commit: 436f0bb9 Date: 2022-07-22T15:47:42Z

$ arduino-cli board list
Port   Protocol Type              Board Name                FQBN             Core       
COM1   serial   Serial Port       Unknown
COM254 serial   Serial Port (USB) Unknown
COM3   serial   Serial Port       Unknown
COM52  serial   Serial Port (USB) Unknown
COM7   serial   Serial Port (USB) Arduino Mega or Mega 2560 arduino:avr:mega arduino:avr

$ rm ~/AppData/Local/Arduino15/packages/builtin/tools/serial-discovery/1.3.2/serial-discovery.exe

$ arduino-cli board list
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xc0000005 code=0x0 addr=0x8 pc=0xdd27cd]

goroutine 40 [running]:
os.(*Process).signal(0xc000150770, {0x196b990, 0x1f386e0})
        C:/Program Files/Go/src/os/exec_windows.go:49 +0x4d
os.(*Process).Signal(...)
        C:/Program Files/Go/src/os/exec.go:138
os.(*Process).kill(...)
        C:/Program Files/Go/src/os/exec_posix.go:68
os.(*Process).Kill(...)
        C:/Program Files/Go/src/os/exec.go:123
github.com/arduino/arduino-cli/executils.(*Process).Kill(...)
        E:/electronics/git-nobackup/arduino-cli/executils/process.go:120
github.com/arduino/arduino-cli/arduino/discovery.(*PluggableDiscovery).killProcess(0xc000985c00)
        E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discovery.go:268 +0xc9
github.com/arduino/arduino-cli/arduino/discovery.(*PluggableDiscovery).Quit(0xc000985c00)
        E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discovery.go:381 +0xde
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).remove(0xc0003782e0, {0x1716256, 0x18})
        E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:82 +0xba
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).RunAll.func1(0xc000985c00)
        E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:131 +0x71
github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).parallelize.func1(0x2e0031002e0032)
        E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:101 +0x68
created by github.com/arduino/arduino-cli/arduino/discovery/discoverymanager.(*DiscoveryManager).parallelize
        E:/electronics/git-nobackup/arduino-cli/arduino/discovery/discoverymanager/discoverymanager.go:99 +0x1c5

[per1234]

#1814 fixed the panic I reported in my previous comment, but it did not resolve the feature request for Arduino CLI to recover from a situation where a discovery executable goes missing.

$ arduino-cli version
arduino-cli  Version: nightly-20220808 Commit: d0b2556 Date: 2022-08-08T01:35:05Z

$ rm ~/.arduino15/packages/builtin/tools/serial-discovery/1.3.2/*

$ arduino-cli board list
Error detecting boards: Error starting board discoveries: [discovery builtin:serial-discovery process not started: fork/exec /home/per/.arduino15/packages/builtin/tools/serial-discovery/1.3.2/serial-discovery: no such file or directory]
No boards found.