get community help for testing your releases - before they ship?

[jerabaul29]

First, many thanks for the great work you do here! I love the idea of the Artemis boards, and I really hope it succeeds. The code here is also improving quickly, which is very nice to see. Your product has really a huge potential.

One thing I have noticed following this project / using it for a few months: there are often issues with new releases, at least on my platform (plain Ubuntu). Recently I have met broken uploaders defaults #310, uploaders without rights #309 , flags that make compilation fail #330 , etc. Small things, but they make using your product impossible without a fix.

• does that indicate that you maybe test / develop on another platform (windows I think I understood?), but not really on Ubuntu (and possibly other platforms)?

• these issues are fine for me; I can downgrade, open an issue here, and you are very helpful. But for "even n00bser than me" users, who may not find the solution here, or may not dare to ask, it may be a real pain and give a bad opinion of your product, which is really sad. In the most extreme case, students with little embedded / programming background may give up altogether quite quick when the "blink sketch uploads" repeatedly fail with the default Arduino IDE package and they don't get it to work after an afternoon or so (I have heard of at least a couple such cases).

Hence my question: should you try to collect a pool of users who can test your planned releases, before you create / package / distribute an official release to the Arduino IDE, so that we can slash at least a few of the bugs on platforms you may not test internally? I would happily participate in a "non committed / informal" way to such testing. Something like:

• you let me (and other people in the pool) know what exact steps to perform to install your new pre-release
• I do a few compilation / upload tests
• I let you know how it went

If you could "collect" a few such pre-testers for Windows / Mac / Linux and avoid shipping broken board versions, I think you could really improve the satisfaction of your users :) .

Due to my work I sometimes have large periods without internet (will have no internet at all for a large part of February for example), so you may want to try to recruit a few testers on each platform. Not sure how this could be organized (sticky issue here? mailing list?).

[paulvha]

agree with this post. count me in...

regards,
Paul

[ArminJo]

The solution is quite simple.
Just use Github actions like the one in https://github.com/ukw100/IRMP/actions/runs/510343640 and you instantly see, which platform fails.
Another example is here #169 (comment)

[Wenn0101]

@jerabaul29 This is a great idea! As a newer member of the Sparkfun team, I can forget about how awesome it can be to be part of an open source community, thanks for the reminder!

As you have properly guessed. My development machine is a Windows machine, and I have don't always have the ability to test all platforms by hand. I have been working on expanding our testing capabilities, but admittedly, functionality has been my focus. As @ArminJo has pointed out, an automated build would catch some of these problem right away. There has been some effort put towards this in the past, but this predated my involvement in the project, and I am yet to commit the time to get this functional.

On future releases, I will put the word out on GitHub (perhaps in issues, or maybe I will try out the new "Discussions" tab) a few days prior to release, so that users can test it out prior to making an Arduino package. For the first couple releases, I will be sure to tag @jerabaul29 and @paulvha, and if participation increases, I may look at formalizing a process around this more!

[jerabaul29]

That sounds excellent @Wenn0101 :) .

Haha yes, I am very keen on the Artemis boards and the environment that comes with. We have several significant pilot open source instrumentation development projects going on at a couple of partner meteorological institutes / universities, and I am trying to push for the Artemis solution. But for this to succeed / my point of view to be accepted, I need to show some good results fast, hence I am very much looking forward to each new release / bug fix / functionality addition (especially if you could get the watchdog out, and some FFT functionality with CMSIS, that would be awesome, my life would get much easier ^^ ) :) . If I can show the results that are expected of the pilot projects, that would be good for all of us - my bosses would be happy, and you may get quite a few "heavy lifting" customers in your ecosystem :) . We just ordered about 15 of the Artemis global trackers for example, but we really need to add some of our own stuff to the firmware :) .

[Wenn0101]

Check out this discussion thread on the next upcoming release.

I don't know if I will do discussions in the future, figured I would give the feature a try.

[andyle9191]

Count me in as well. I have been following each Github update regularly. I'm currently developing a product that I will likely end up with an Artemis. Hardware-wise it fits perfectly and will end up using a lot of its features. So far I've been resisting using any non-Arduino concepts but it's only a matter of time before I want to utilize v2.x libraries (RTOS, Arduino's_BLE, file system) and I can report back if problems are found. P.S I'm on a Mac

[adamgarbo]

I'd also be more than happy to contribute wherever possible and to provide some feedback from a macOS perspective. Many thanks, @jerabaul29 for creating this issue!

[jerabaul29]

😊👌👍

[marsfan]

@Wenn0101 If there is still plants for this, I would be on board.