arduino-cli upload not working but works in Arduino IDE

[afoxinsocks]

Subject of the issue

Attempting to use the latest 0.7.1 arduino-cli on macOS with a SparkFun RedBoard Artemis Nano fails after detecting bootloader.

Your workbench

• What platform are you using?
  macOS 10.14

• What version of the device are you using? Is there a firmware version?
  SparkFun RedBoard Artemis Nano with SVL Bootloader 5 programmed from the 1.0.23 library.

• How is the device wired to your platform?
  USB C cable

• How is everything being powered?

• Are there any additional details that may help us help you?
  Uploading works fine in the Arduino IDE

Steps to reproduce

Tell us how to reproduce this issue. Please post stripped down example code demonstrating your issue to a gist.

arduino-cli compile -u -t -b SparkFun:apollo3:amap3nano -p /dev/cu.wchusbserial1420 <sketch>

Expected behaviour

The Arduino IDE functions correctly and outputs the following:

Sketch uses 14284 bytes (1%) of program storage space. Maximum is 960000 bytes.


Artemis SVL Bootloader
Got SVL Bootloader Version: 5
[##################################################]Upload complete

Actual behaviour

Attempting to do the same thing with arduino-cli from a Terminal window outputs the following:

Sketch uses 14284 bytes (1%) of program storage space. Maximum is 960000 bytes.


Artemis SVL Bootloader
Got SVL Bootloader Version: 5
/dev/cu.wchusbserial1420 is currently open. Please close any other terminal programs that may be using /dev/cu.wchusbserial1420 and try again.

Note: Closing the Arduino IDE before trying arduino-cli does not solve the problem, nor does rebooting. Sudo also doesn't help.

[oclyke]

@afoxinsocks Thanks for the issue.

In the past the upload script had a bug that caused any error to report as 'com port is currently open' however that was fixed. This is a little odd to me - and also new territory because I am not familiar with the CLI.

Can you answer a few more questions?

1. Did the CLI work at any point before? If so can you recall what conditions (CLI version, Apollo3 core version, bootloader version etc...)?
2. Does uploading to your board work in the regular IDE with all the latest software?

[frinkahedron]

@oclyke I'd check tools/artemis/artemis_svl.py again, it looks like a commit (977cad9) on December 23rd overwrote the previous fix, I was about to write an issue asking if this was on purpose and I saw this issue. That said its probably just half the issue, I need to investigate more if arduino-cli is moving the bin file correctly (doesn't on my machine).

[frinkahedron]

@afoxinsocks
This is a guess based on my current experience so I may be wrong, but I get this behavior due to two things.

1. The root issue is likely that a .bin file is not showing up in your sketch directory, take a look, your probably only getting a .axf file.
2. The script oclyke mentions had an issue where you get a misleading error message if it can't find the .bin file. It was fixed but looks like it may have regressed.

So try the following as a work around if you verify you actually don't have a .bin file in your sketch directory (if you do then I am wrong and ignore this)
Try one of the following:

1. Copy the .bin in manually matching the name of the .axf file (except having a .bin extension). You can find it in your OS temp directories being used by arduino-cli, if unsure rerun the compile command but add --show-properties and you will see a bunch of key=value pairs, look for "build.path" that will probably be the directory to look in.
2. you could generate the .bin yourself afterwards using arm-none-eabi-objcopy

Once you have the .bin file if the arduino-cli is still problematic you can also just run the uploader tool directly, on my machine (with arduino-cli) it was found at ~/.arduino15/packages/SparkFun/hardware/apollo3/1.0.28/tools/artemis/artemis_svl.py

(or)
I have not tried this yet, but seems like you could also edit the boards.txt file and modify the ".build.preferred_export_format=axf" line for the desired board to be "=bin". Mine was found at ".arduino15/packages/SparkFun/hardware/apollo3/1.0.28"

The root cause of not getting the bin file in the sketch directory appears to be assumptions by the arduino-cli tool on what files it should copy and only appears to allow you to override that for a single file and in this case that is the file specified by the "{boardname}.build.preferred_export_format=axf" in the boards.txt file and "recipe.output.tmp_file={build.project_name}.{build.preferred_export_format}" parameter in the platform.txt file for the sparkfun core.

I just started looking at his today after having the same issue so I have not decided the best way to fix it yet (I am trying to use arduino-cli instead of the GUI), I am new to arduino and arduino-cli so there may obvious fixes to those with more experience.

Hope that is of some use

[frinkahedron]

@afoxinsocks In case you run into this,
Turns out (at least on my machine) there is a third factor.
Even with a properly names/located bin file the upload still failed for me (different errors about receiving packets), turns out arduino-cli was using the default (highest) baud rate for the board and that was causing the issue.

To fix that I overrode the svl baud rate using the fqbn string for the arduino-cli command so something like
--fqbn SparkFun:apollo3:amap3atp:svl_baud=115200 (in the case of the atp board)

[oclyke]

@frinkahedron thanks for investigating this further.

1. You're correct - the error message fix was accidentally lost. I'll make this into its own issue so that it can be fixed in the next release.
2. Can you tell me more about the CLI? Specifically it seems odd to me that it would rely on a .bin existing in the sketch folder. How come the CLI doesn't operate similarly to the IDE and upload the generated .bin from the temporary build folder?
3. You are correct about having the option to change your desired output file format. An improvement we could make is to provide menu options to change that without having to edit platform.txt.

[frinkahedron]

@oclyke
I'm just starting to use the CLI so I had the same question as to why its behaving different for me from IDE, etc. I'll find some time this week to dig into it more, could be a combination of things or me just missing something obvious. I did find where they copy things over to the src dir in their code but have not compared to IDE yet.

[frinkahedron]

@oclyke
I looked into this a bit more (item 2 on your question) and I think it basically comes down to the cli project being alpha still and its a separate implementation not an extension of the IDE code. While the metadata like the board and platform files are common how they interact with it is not consistent between the two currently.

If useful I can detail more info on how their logic is currently implemented but don't want to muddy this issue too much further if not desired (since it might not even be directly related to the original issue described). Here is a quick summary of the cli though (based on my setup):

• By default the compile command uses a temp build folder but when complete copies the file specified by recipe.output.tmp_file (of platform.txt) to the sketch's directory.
• The upload command looks in the sketch folder for the file specified by recipe.output.tmp_file of platform.txt. This happens to be .axf file based on current values in boards.txt for the artemis boards but could be changed to .bin and then things will work because .bin will then be copied into sketch folder where the cli upload expects it by default
• Using --build-dir and --build-cache options to the compile command would allow you to control the location of the tmp dir and then you could use upload command --input to specify the input file but because of how things were implemented (last time I checked) it would still require modifying some of the Sparkfun parameters in platform.txt or boards.txt to work as it appends data to the end of that parameter which would prevent the correct file from being found in the tmp build directory.

Ultimately it would not be too hard to fix these issues but the cleanest solution would seem to make modifications to arduino-cli to implement consistent behavior between the cli and IDE versions (if that project would entertain that, and maybe they still will since in alpha).

For now I am just using a make file to manipulate things to work without modifying either arduino-cli or the sparkfun core files while arduino-cli matures. I can share that if anyone finds it useful, I basically just modified the example file here: https://learn.sparkfun.com/tutorials/efficient-arduino-programming-with-arduino-cli-and-visual-studio-code/equipping-vs-code-with-arduino-cli to copy over the .bin manually using the proper name the upload command expects

[oclyke]

@frinkahedron Awesome! Good to know. We definitely want to keep an eye on where Arduino-CLI is headed because I hear the next big thing from Arduino (i.e. the next IDE) is going to be built on top of the CLI.

I will open an issue regarding the default output format. I had chosen .axf because it is much more useful for debugging. I did not anticipate this problem. I believe it is possible, however, to make the default .bin so that it will work with the CLI and provide a menu option to allow users to select .axf without manually editing core files.

[oclyke]

@frinkahedron can we close this issue in favor of #124?

[frinkahedron]

Since it was opened by afoxinsocks I would say we should let them decide that, its possible this is not the root cause of their problem, but otherwise I have no issue with that.

[afoxinsocks]

Fine with me. Thanks!