Description
The "Windows local development validation" pipeline currently takes close to two hours to complete. We should consider if there are any high-cost+low-value validation that we can remove to make this pipeline more efficient.
For example, we might be able to optimize or remove the following steps:
- Build (Debug): ~10 min
- Run project template tests: ~50 minutes
Additional context
The initial motivation for adding this pipeline was that our project template tests kept getting broken by changes that passed in CI but silently broke something because either:
- An already-quarantined tests got broken to a more severe degree, but this break was invisible
- A scenario was only used during local development (e.g., testing a project template manually) and it didn't get validated in CI at all
So, removing the project template test stage in this pipeline might seem to counteract that initial goal. However, we recently added another pipeline that runs a test script to create a project from a template and check that it builds successfully. This probably covers the majority of what we're trying to do, minus the actually running the template.
As an alternative to removing the project template test step entirely, we could either add a test filter on the step to only test a small subset of project template tests, or we could add a new test script that runs the built project and does some basic validation on it.