python.exe": file does not exist

[Juppit]

Basic Infos

• This issue complies with the issue POLICY doc.
• I have read the documentation at readthedocs and the issue is not addressed there.
• I have tested that the issue is present in current master branch (aka latest git).
• I have searched the issue tracker for a similar issue.
• I have filled out all fields below.

Platform

• Hardware: [ESP12]
• Core Version: [2.6.0-dev-nightly+20190315]
• Development Env: [Arduino IDE 1.8.8]
• Operating System: [Windows]

Settings in IDE

• Module: [LOLIN(Wemos) D1 mini r2]
• Flash Size: [4MB/1MB]
• lwip Variant: [|v2 Lower Memory]
• CPU Frequency: [80Mhz]
• Upload Using: [SERIAL]
• Upload Speed: [921600]

Problem Description

Building any sketch I get the error:
"C:\\Users\\Peter\\AppData\\Local\\Arduino15\\packages\\esp8266\\hardware\\esp8266\\2.6.0-dev-nightly+20190315/tools/python/python.exe": file does not exist

The file python.exe is not in the tools directory within 2.6.0-dev..., it is in the directory
C:\Users\Peter\AppData\Local\Arduino15\packages\esp8266\tools\python

My build steps from Cygwin on Windows 10:

1. clone https://github.com/esp8266/Arduino.git
2. cd esp8266/package
3. ./build_boards_manager_package.sh
4. cp package_esp8266com_index.json from versions/2.6.0-dev... to versions/package_esp8266com_index.json
5. setup the boardmanager URL in the Arduino-IDE to http://localhost:8000/versions/package_esp8266com_index.json
6. start a local http-server to delever the package to the Arduino-IDE
7. start the boardmanager from Arduino tools-menu and download the esp8266 version 2.6.0-dev...

One note: to build step 3, there is jq.exe in version 1.6 necessary

[Juppit]

After the installation python.exe is hier:
C:\Users\Peter\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.0-dev-nightly+20190315\tools\python\3.7.2-post1\python.exe

[earlephilhower]

Ah, thanks for trying this out. So, in summary, python.exe location is busted when installed using a packaged JSON file. That was my worry as well.

It should just require adjusting to the "proper" Arduino macros inside the boards.txt. I'll look at it this weekend and see what I can find.

[Juppit]

@earlephilhower Just to let you know, if I move the content of 2.6.0-dev-...\tools\python\3.7.2-post1\* to 2.6.0-dev-...\tools\python\ then everything works fine.

[JAndrassy]

the path to a tool installed in Arduino IDE is runtime.tools.<name>.path. the variable is provided by IDE (arduino builder to be exact). <name> is the tool's folder name in Arduino15\packages\esp8266\tools

https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5-3rd-party-Hardware-specification#tools

[earlephilhower]

I need to make a fake Python package for Linux as well. Arduino complains if a tool is missing:

 Tool python is not available for your operating system.
java.lang.RuntimeException: java.lang.Exception: Tool python is not available for your operating system.
	at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onInstallPressed$2(ContributionManagerUI.java:175)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.Exception: Tool python is not available for your operating system.
	at cc.arduino.contributions.packages.ContributionInstaller.install(ContributionInstaller.java:85)
	at cc.arduino.contributions.packages.ui.ContributionManagerUI.lambda$onInstallPressed$2(ContributionManagerUI.java:172)
	... 1 more

[earlephilhower]

@Juppit , thanks for the easy reproduction steps. I've tried on my Windows VM and Linux host and all seems to be working with the above PR. Could you give it a try?

[Juppit]

@earlephilhower
unluckily ...
I find python.exe now here:
...\Arduino15\packages\esp8266\tools\python\3.7.2-post1\python.exe

I'll try to find the right changes for build_boards_manager_package script if the next few days are not too busy for me.

[Juppit]

Upps,
I thought the changes were already merged.
I'll try ...

[Juppit]

@earlephilhower
With your changes I can compile, but the upload does not succeed:
I get these error message:
{runtime.tools.python.windows.path}/python.exe C:\Users\Peter\AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.6.0-dev-nightly+20190320/tools/upload.py --chip esp8266 --port COM5 --baud 921600 --trace version --end --chip esp8266 --port COM5 --baud 921600 --trace write_flash 0x0 C:\Users\Peter\AppData\Local\Temp\arduino_build_390263/Sketch.ino.bin --end

java.io.IOException: Cannot run program "{runtime.tools.python.windows.path}/python.exe": CreateProcess error=2
this means "file not found"

[earlephilhower]

Thanks, @Juppit . Uploading was one thing I did not try on Windows, and it was definitely busted.

I've done 4 tests building and uploading (win/git, win/json, lin/git, lin/json) and it seems to be running with the latest version of the PR. Basically the upload stage cannot see some of the variables in the rest of the file, just to be different, so we need to swap paths there like we do in the GCC binary section between GIT and JSON.

If you've got another few mins, can you check yourself? It gets confusing swapping between two systems and uninstalling/renaming/etc. to test this...

[Juppit]

@earlephilhower
Well, it's confusing, even on just one system, but I should have done it on my side.
Compile and upload succeed now.
But there is something else, esptyool.py is now verbose:

esptool.py v2.6
2.6
esptool.py v2.6
Serial port COM5
Connecting....
Chip is ESP8266EX
Features: WiFi
MAC: cc:50:e3:0f:25:98
Uploading stub...
Running stub...
Stub running...
Changing baud rate to 460800
Changed.
Configuring flash size...
Auto-detected Flash size: 4MB
Compressed 355680 bytes to 251367...

[earlephilhower]

That's always output, not sure what to say. Verbose actually dumps the entire protocol with byte-by-byte hex dumps of the serial port/etc. You may be thinking of the older esptool-ck.exe output, which is pretty much quiet except for the "..."s as it uploads.