Distinguish between unit test compilation and "normal" compilation

[ianfixes]

Feature Request

via https://github.com/ianfixes/arduino_ci/issues/150#issuecomment-689887131

I find that I'd like a way to distinguish between when code is compiled from the Arduino IDE and when code is compiled from arduino_ci. Ideal would be something like #define ARDUINO_CI but the closest I've come is MOCK_PINS_COUNT

Initial thoughts

There is some ability to do this already, as a per-project workaround, by copying platform definitions from default.yml and adding to the defines section of a platform:

# copied from default.yml in september 2020
platforms:
  uno:
    board: arduino:avr:uno
    package: arduino:avr
    gcc:
      features:
      defines:
        - __AVR_ATmega328P__

        # This is the workaround described above
        - ARDUINO_CI
      warnings:
      flags:

Since the platform definitions' gcc section applies only to unit testing (i.e. it's not AVR compilation), this is safe. But possibly not ideal.

Discussion points

What are best practices for injecting "evidence" of test platforms into compilation? For example, should we copy the pattern used by the ARDUINO constant, and if so, what would the numbers represent?

[ianfixes]

My main concern is how likely it might be that someone else has defined ARDUINO_CI (something close enough to the name of the platform itself that I'd be worried about having it defined by the core project in the future). Given #150 and the fact that multiple libraries using arduino_ci might exist in the wild (all with their own different versions), they will only work properly together if they all respond to the same constants.

[per1234]

Follow the same precedent Arduino uses by prefixing global macro names with the organization name. This creates a "namespace" of sorts for your macros to avoid collisions: IANFIXES_ARDUINO_CI

[ianfixes]

I guess that keeps me safe if the project gets forked or adopted elsewhere 😅

Is using my github name appropriate enough for an Arduino project? I feel like it would be a bad mnemonic to have the constant be "the author's github name, underscore arduino, underscore CI".

I'm wondering if I should be exploring names more indicative of the idea that "this is not real hardware, this is a unit test".

[matthijskooijman]

I'm wondering if I should be exploring names more indicative of the idea that "this is not real hardware, this is a unit test".

That also sounds like a good idea to me, though it would (maybe) not allow code to distinguish whether they're running under arduino_ci, or maybe some alternative non-hardware Arduino core that uses the same non-hardware define.

Also, I've been thinking about this same thing for a while (partially for such a define, and partially for finding a name for such a virtual arduino core), and I'm not sure what terminology would be appropriate here. Things I considered are "Running on host" (but that's vague), "Running on x86" (but it might also be amd64 or arm/rpi), "Running on PC" (but how about Macs?). Writing this, maybe "Virtual" is a useful concept here, since all of these are somewhat emulating a virtual Arduino (in the sense that e.g. digitalWrite() does not write to actual hardware pins, but virtual ones. This would even allow distinguishing from proper Arduino cores that run on e.g. a Rpi (not sure if they exist, but they could exist and also compile to a regular Linux executable, where digitalWrite() controls actual Rpi pins), which would not be virtual.

[ianfixes]

My best idea so far is ARDUINO_CI_UNITTEST_ENVIRONMENT which is long enough that it probably won't cause a collision, & is descriptive as to its purpose and meaning. I'll also probably assign it some number value, and come up with a vague notion of what that number means & more importantly when it should be incremented.

[jgfoster]

I'm still using MOCK_PINS_COUNT as my flag and find that somewhat ugly.

Follow the same precedent Arduino uses by prefixing global macro names with the organization name.

Can we just go with our organization name, ARDUINO_CI?

[ianfixes]

This issue is old enough that it predates the move from my ianfixes namespace to Arduino-CI 😅

After re-reading this thread, I think I see what was bothering me about this earlier: ARDUINO_CI is/was generic, yes, but the real problem is that it didn't convey the idea that "hey, this is not the real Arduino compiler nor the real Arduino system libraries".

So, in order of preference, here's how I'd like to name the constant such that it signals to the environment that (1) the "Arduino CI" system is running things, and (2) compilation mocks are in effect. In order of preference:

• ARDUINO_CI_COMPILATION_MOCKS
• ARDUINO_CI_UNITTEST_MOCKS
• ARDUINO_CI_MOCKS
• ARDUINO_CI_MOCKED_LIBRARIES

Any objections to these, and/or is there a better way to convey "Arduino" + "Arduino-CI" + "stuff is mocked" + "this is only for unit test compilation" to the user?

[jgfoster]

For me, ARDUINO_CI adequately communicates the mocked/testing situation. See here for a sample usage. But I can live with any of your proposals.