Property not evaluated at the upload step

[fpistm]

Hi,
Testing this PR stm32duino/Arduino_Core_STM32#238
to add BMP upload method show that some properties are not properly evaluated at upload stage.
It seems to be a regression as it was functional before using @rogerclarkmelbourne:
https://github.com/rogerclarkmelbourne/Arduino_STM32/blob/8fa6bb45960097be75b6df6fc08705c85d4e4a09/STM32F1/platform.txt#L148

# blackmagic upload for generic STM32
tools.bmp_upload.cmd=arm-none-eabi-gdb
tools.bmp_upload.path={runtime.tools.arm-none-eabi-gcc-6-2017-q2-update.path}/bin/
tools.bmp_upload.upload.speed=230400
tools.bmp_upload.upload.params.verbose=-batch
tools.bmp_upload.upload.params.quiet=--batch-silent
tools.bmp_upload.upload.pattern="{path}{cmd}" -nx {upload.verbose} -ex "set confirm off" -ex "target extended-remote {serial.port}"  -ex "monitor swdp_scan"  -ex "attach 1" -ex "load" -ex "compare-sections" -ex "kill" "{build.path}/{build.project_name}.elf"

{runtime.tools.arm-none-eabi-gcc-6-2017-q2-update.path} is not expanded here.

Tested with Arduino-1.8.5 on
Windows:

{runtime.tools.arm-none-eabi-gcc-6-2017-q2-update.path}/bin/arm-none-eabi-gdb -nx -batch -ex set confirm off -ex target extended-remote COM1 -ex monitor swdp_scan -ex attach 1 -ex load -ex compare-sections -ex kill C:\Users\frq92243\AppData\Local\Temp\arduino_build_395606/sketch_may21a.ino.elf 
java.io.IOException: Cannot run program "{runtime.tools.arm-none-eabi-gcc-6-2017-q2-update.path}/bin/arm-none-eabi-gdb": CreateProcess error=2, The system cannot find the file specified
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at processing.app.helpers.ProcessUtils.exec(ProcessUtils.java:26)
	at cc.arduino.packages.Uploader.executeUploadCommand(Uploader.java:129)
	at cc.arduino.packages.uploaders.SerialUploader.uploadUsingPreferences(SerialUploader.java:207)
	at cc.arduino.UploaderUtils.upload(UploaderUtils.java:78)
	at processing.app.SketchController.upload(SketchController.java:713)
	at processing.app.SketchController.exportApplet(SketchController.java:686)
	at processing.app.Editor$DefaultExportHandler.run(Editor.java:2168)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
	at java.lang.ProcessImpl.create(Native Method)
	at java.lang.ProcessImpl.<init>(ProcessImpl.java:386)
	at java.lang.ProcessImpl.start(ProcessImpl.java:137)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 8 more
An error occurred while uploading the sketch

Linux:

{runtime.tools.arm-none-eabi-gcc-6-2017-q2-update.path}/bin/arm-none-eabi-gdb -nx -batch -ex set confirm off -ex target extended-remote /dev/ttyACM0 -ex monitor swdp_scan -ex attach 1 -ex load -ex compare-sections -ex kill /tmp/arduino_build_762465/Blink.ino.elf 
java.io.IOException: Cannot run program "{runtime.tools.arm-none-eabi-gcc-6-2017-q2-update.path}/bin/arm-none-eabi-gdb": error=2, No such file or directory
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at java.lang.Runtime.exec(Runtime.java:620)
	at java.lang.Runtime.exec(Runtime.java:485)
	at processing.app.helpers.ProcessUtils.exec(ProcessUtils.java:11)
	at cc.arduino.packages.Uploader.executeUploadCommand(Uploader.java:129)
	at cc.arduino.packages.uploaders.SerialUploader.uploadUsingPreferences(SerialUploader.java:207)
	at cc.arduino.UploaderUtils.upload(UploaderUtils.java:78)
	at processing.app.SketchController.upload(SketchController.java:713)
	at processing.app.SketchController.exportApplet(SketchController.java:686)
	at processing.app.Editor$DefaultExportHandler.run(Editor.java:2168)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: error=2, No such file or directory
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 10 more
An error occurred while uploading the sketch

[facchinm]

Hi @fpistm ,
is arm-none-eabi-gcc-6-2017-q2-update specified in a json? There shouldn't be any problem in expanding it if the system is able to resolve the qualified name

[fpistm]

Hi @facchinm,
this is the version used to build the core. It is well expanded for the build.
https://github.com/stm32duino/Arduino_Core_STM32/blob/dcf191fe917fd121e0653d1d4082b2e6ee3e5b0b/platform.txt#L22
(Note that I've also try to use {compiler.path} without any success)

It is provided thanks the package_stm_index.json:
https://github.com/stm32duino/BoardManagerFiles/blob/0f67f99f539e4f4de125b9b9d785b5a1acf97843/STM32/package_stm_index.json#L392
https://github.com/stm32duino/BoardManagerFiles/blob/0f67f99f539e4f4de125b9b9d785b5a1acf97843/STM32/package_stm_index.json#L410

[fpistm]

I made some more test, it seems linked to my environment.
As I use a clone of the core repo in hardware folder, it do not linked properly the version.
But it's ok for the build part so why not the same for upload? This mean properties value are not populated ?
Anyway, it should be fine to be able using a custom property. Here, {compiler.path} is set to the path required, so it should be fine to be able to use it to avoid duplicated lines and any misalignment if the gcc version change for example.

This how I set up my env:
Arduino with portable folder.
Then I install the latest STM32 core thanks the board manager (here 1.2.0) which install the core, the CMSIS, the tools and the arm gcc toolchain.
Then I remove in portable/packages/STM32/hardware/stm32/1.2.0 the version directory of the core
Finally, I clone my core in hardware/STM32/stm32

[facchinm]

Compilation variables are resolved by the builder while upload ones are still resolved by the Java IDE. Probably the implementation is slightly different and in one case the folder is being ignored.
I'll take a closer look soonish 🙂

[fpistm]

Thanks a lot @facchinm ;)

[facchinm]

I found what is going on: since you removed the core installed via board manager, the Java IDE avoids creating any cross reference with the json (so only {runtime.tools.arm-none-eabi-gcc} is available, not the "decorated" one).
The builder doesn't know about the json and happily adds every path under ~/.arduino15/packages to the runtime variables.
The first behaviour is the "correct" one, since the cross reference is specified by the core author, not guessed by a tool 😄
So, I'm closing this one as "not a bug". For future reference, when working with git cores installed in the sketchbook, don't remove the one installed via board manager 😉

[fpistm]

Thanks @facchinm for the quick and precise answer.

So, I'm closing this one as "not a bug". For future reference, when working with git cores installed in the sketchbook, don't remove the one installed via board manager wink

Yes but if I do not delete it it appears twice :'(

[facchinm]

I have to deal with it daily, being a developer has these little downsides 😜
Jokes apart, keeping a released version installed is incredibly useful to quickly check for regressions, at least for me :)