Unit tests produce the error "Some of your tests did a full page reload!" after Chrome update

[philmayfield]

Which @angular/* package(s) are the source of the bug?

core

Is this a regression?

No

Description

I don't believe is an Angular issue, but I'm creating this report for clarity and to increase visibility.

Since updating to Chrome ~128.x running unit tests produce an error in spite of all tests passing. To reproduce you can simply create a new project and run ng test.

Please provide a link to a minimal reproduction of the bug

No response

Please provide the exception or error you saw

Chrome Headless 128.0.0.0 (Mac OS 10.15.7) ERROR
  Some of your tests did a full page reload!
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 4129 of 4129 SUCCESS (18.521 secs / 17.638 secs)

Please provide the environment you discovered this bug in (run ng version)

Angular CLI: 17.3.8
Node: 18.20.4
Package Manager: npm 10.7.0
OS: darwin arm64

Angular: 17.3.11
... animations, common, compiler, compiler-cli, core, forms
... language-service, localize, platform-browser
... platform-browser-dynamic, router

Package                            Version
------------------------------------------------------------
@angular-devkit/architect          0.1703.8
@angular-devkit/build-angular      17.3.8
@angular-devkit/core               17.3.8
@angular-devkit/schematics         17.3.8
@angular-devkit/schematics-cli     17.3.8
@angular/cdk                       17.3.10
@angular/cli                       17.3.8
@angular/flex-layout               15.0.0-beta.42
@angular/material                  17.3.10
@angular/material-moment-adapter   17.3.10
@schematics/angular                17.3.8
ng-packagr                         17.3.0
rxjs                               7.8.1
typescript                         5.3.3
zone.js                            0.14.7

Anything else?

I tested it out on new Angular 17 and 18 projects using node 18 and 20 and get the same results for all of them. I was able to verify that upon reverting to Chrome 127.x the error is not produced.

[alemp]

Same happened here after update Chrome to v128.0.0.0.

Angular CLI: 17.3.8
Node: 18.20.3
Package Manager: npm 10.7.0
OS: darwin arm64

Angular: 17.3.12
... animations, common, compiler, compiler-cli, core, forms
... platform-browser, platform-browser-dynamic, router

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1703.8
@angular-devkit/build-angular   17.3.8
@angular-devkit/core            17.3.8
@angular-devkit/schematics      17.3.8
@angular/cdk                    17.3.10
@angular/cli                    17.3.8
@angular/material               17.3.10
@schematics/angular             17.3.8
rxjs                            7.8.1
typescript                      5.4.5
zone.js                         0.14.10

[freon27]

I also have this issue. I'm only commenting to add that I ONLY see the problem in non-watch mode : ng test --watch=false

[OPuder]

The same thing happened here after updating Chrome to v128.0.0.0.

[philmayfield]

I ONLY see the problem in non-watch mode : ng test --watch=false

I noticed that too, but I think once you stop watching it'll spit out the error.

[rdamazio]

Also seeing this, and it's not a specific test that fails:

Angular CLI: 18.2.1
Node: 22.7.0
Package Manager: npm 10.8.2
OS: darwin x64

Angular: 18.2.1
... animations, cdk, cli, common, compiler, compiler-cli, core
... forms, material, platform-browser, platform-browser-dynamic
... platform-server, router, ssr

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1802.1
@angular-devkit/build-angular   18.2.1
@angular-devkit/core            18.2.1
@angular-devkit/schematics      18.2.1
@schematics/angular             18.2.1
rxjs                            7.8.1
typescript                      5.5.4
zone.js                         0.14.10

Test output:

> ng test --watch=false --browsers=ChromeHeadless

✔ Browser application bundle generation complete.
23 08 2024 15:49:17.645:INFO [karma-server]: Karma v6.4.4 server started at http://localhost:9876/
23 08 2024 15:49:17.647:INFO [launcher]: Launching browsers ChromeHeadless with concurrency unlimited
23 08 2024 15:49:17.651:INFO [launcher]: Starting browser ChromeHeadless
23 08 2024 15:49:19.042:INFO [Chrome Headless 128.0.0.0 (Mac OS 10.15.7)]: Connected on socket 0lhJDWyRZAKUz0g-AAAB with id 93949148
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 0 of 36 SUCCESS (0 secs / 0 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 0 of 36 SUCCESS (0 secs / 0 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 0 of 36 SUCCESS (0 secs / 0 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 0 of 36 SUCCESS (0 secs / 0 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 0 of 36 SUCCESS (0 secs / 0 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 0 of 36 SUCCESS (0 secs / 0 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 0 of 36 SUCCESS (0 secs / 0 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 36 of 36 SUCCESS (0.536 secs / 0.5 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7) ERROR
  Some of your tests did a full page reload!
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 36 of 36 ERROR (0.536 secs / 0.5 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7) ERROR
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 36 of 36 ERROR (0.55 secs / 0.5 secs)

[andonary]

Similar issue here, does not appear on watch=true. I can't find what is the root cause of it.

Angular CLI: 18.0.4
Node: 22.2.0
Package Manager: npm 10.7.0
OS: darwin arm64

Angular: 
... 

Package                      Version
------------------------------------------------------
@angular-devkit/architect    0.1800.4
@angular-devkit/core         18.0.4
@angular-devkit/schematics   18.0.4
@schematics/angular          18.0.4

> ng test --no-watch --no-progress --browsers=ChromeHeadless

26 08 2024 13:56:56.078:INFO [karma-server]: Karma v6.4.3 server started at http://localhost:9876/
26 08 2024 13:56:56.079:INFO [launcher]: Launching browsers ChromeHeadless with concurrency unlimited
26 08 2024 13:56:56.081:INFO [launcher]: Starting browser ChromeHeadless
26 08 2024 13:56:56.739:INFO [Chrome Headless 128.0.0.0 (Mac OS 10.15.7)]: Connected on socket SRHYT2w467iGirpgAAAB with id 23475697
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 10 of 73 SUCCESS (0 secs / 0.045 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 73 of 73 SUCCESS (0.191 secs / 0.173 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7) ERROR
  Some of your tests did a full page reload!
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 73 of 73 ERROR (0.191 secs / 0.173 secs)
Chrome Headless 128.0.0.0 (Mac OS 10.15.7) ERROR
Chrome Headless 128.0.0.0 (Mac OS 10.15.7): Executed 73 of 73 ERROR (0.195 secs / 0.173 secs)

[Blackbaud-SteveBrush]

Related? karma-runner/karma#3560 (comment)

I set clearContext: config.singleRun in my local karma.config and the error goes away...

[rachcampitos]

Related? karma-runner/karma#3560 (comment)

I set clearContext: config.singleRun in my local karma.config and the error goes away...

In my case I did the same but still getting the error

[dgp1130]

I'm chatting with some Karma folks to see if we can understand what's going on here. Unfortunately we're not able to reproduce the exact issue, so I have a few questions someone can maybe help out with:

1. Can anyone provide a minimal example? The OP suggests "To reproduce you can simply create a new project and run ng test.", though that's not sufficient for me. I'm wondering if anyone can confirm on their end that this is a sufficient reproduction, or if other requirements are necessary?
2. All the reports I see here list Chrome Headless. Is anyone able to reproduce with a "headfull" test?
3. Related, is anyone able to reproduce with --headless=new? I suspect using the ChromeHeadless browser likely uses the old version of --headless in Chrome and I'm wondering if the issue might be related to that. I think testing both of these require specifying a "custom browser" in your Karma config.
4. All the reports I see here are also darwin arm64. I happen to be on Linux, so I'm wondering if anyone can reproduce on a non-Mac to at least rule out that particular aspect as a key requirement.
5. Do tests actually fail because of this error (exit code != 0) or is this just a spurious warning which doesn't actually affect the result of the test?

[Stargator]

@dgp1130
I'll take a stab at answering some of these from what I see:

Chrome version: 128.0.6613.85
@angular/core: 17.3.10
@angular/cli: 17.3.8

1. Can anyone provide a minimal example? The OP suggests "To reproduce you can simply create a new project and run ng test.", though that's not sufficient for me. I'm wondering if anyone can confirm on their end that this is a sufficient reproduction, or if other requirements are necessary?

I cannot do this. I just got the error and I am rushing to deliver a product. Maybe if no one else does, I can follow-up later on Tues.

2. All the reports I see here list Chrome Headless. Is anyone able to reproduce with a "headfull" test?

I have ran ng test and I do not get the error. Then I run the tests in headless mode and I see the error.

3. Related, is anyone able to reproduce with --headless=new? I suspect using the ChromeHeadless browser likely uses the old version of --headless in Chrome and I'm wondering if the issue might be related to that. I think testing both of these require specifying a "custom browser" in your Karma config.

I added --headless=new to my karma.config.js and the error persists

4. All the reports I see here are also darwin arm64. I happen to be on Linux, so I'm wondering if anyone can reproduce on a non-Mac to at least rule out that particular aspect as a key requirement.

I am also on a darwin arm64, so I cannot help there :(

5. Do tests actually fail because of this error (exit code != 0) or is this just a spurious warning which doesn't actually affect the result of the test?

1. I ran echo $? after running the headless tests and it responded with 0. So maybe it wouldn't cause a failure.
2. I ran the tests in our pipeline, but it currently uses Chrome 124.0.6367.201 and it didn't report any error.

[andonary]

I have ng test on my pre-commit using husky. If it throws an code != 0 I wouldn't able to commit and I can.

I'll watch for a minimal example but same as Stargator I get this error on a enterprise product I cannot show the code.

[freon27]

It caused my pre-push hook to fail and I had to switch off the hooks to be able to push so for us it appears to be actually causing a non-zero exit.