build: do not consider e2e test targets as flaky locally (#20371)

Running e2e test targets locally means that Bazel will re-run
them a second time if they fail. This slows-down local development
as there is no way to see the test failure before Bazel completed
the second run. We should not consider e2e tests as flaky locally.

Developers could re-run the test themselves if they see it being
flaky. Also flaky e2e tests only seem to happen if lots of e2e
tests execute concurrently. This is not often the case locally.

Author: devversion

.circleci/config.yml

@@ -244,7 +244,10 @@ jobs:
       - *yarn_install
       - *setup_bazel_binary
 
-      - run: bazel test src/... --build_tag_filters=e2e --test_tag_filters=e2e --build_tests_only
+      # Run e2e tests. Note that protractor test targets are flaky sometimes, so we run them
+      # with flaky test attempts. This means that Bazel will re-run a failed e2e test target
+      # a second time to ensure it's a real test failure. This improves CI stability.
+      - run: yarn e2e --flaky_test_attempts=2
       - *slack_notify_on_failure
 
   # ------------------------------------------------------------------------------------------

package.json

@@ -21,7 +21,7 @@
     "test-local": "yarn -s test --local",
     "test-firefox": "yarn -s test --firefox",
     "lint": "yarn -s tslint && yarn -s stylelint && yarn -s ownerslint && yarn -s ng-dev format changed --check",
-    "e2e": "bazel test //src/... --test_tag_filters=e2e",
+    "e2e": "bazel test //src/... --build_tag_filters=e2e --test_tag_filters=e2e --build_tests_only",
     "deploy-dev-app": "node ./scripts/deploy-dev-app.js",
     "breaking-changes": "ts-node --project scripts/tsconfig.json scripts/breaking-changes.ts",
     "gulp": "gulp",

tools/defaults.bzl

@@ -198,12 +198,8 @@ def karma_web_test_suite(name, **kwargs):
         **kwargs
     )
 
-# Protractor web test targets are flaky by default as the browser can sometimes
-# crash (e.g. due to too much concurrency). Passing the "flaky" flag ensures that
-# Bazel detects flaky tests and re-runs these a second time in case of a flake.
-def protractor_web_test_suite(flaky = True, **kwargs):
+def protractor_web_test_suite(**kwargs):
     _protractor_web_test_suite(
-        flaky = flaky,
         browsers = ["@npm_angular_dev_infra_private//browsers/chromium:chromium"],
         **kwargs
     )