Regression: upgrading from 6.0.7 to 6.0.8 makes test fail

[jnizet]

A test that has been passing for months, with many different versions of Angular and Angular CLI, fails every time it's run afer an upgrade from cli 6.0.7 and devkit 0.6.7 to cli 6.0.8 and devkit 0.6.8, for a reason that I can't explain.

Bug Report or Feature Request (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request

Area

- [x] devkit
- [ ] schematics

Versions

Node v8.11.1
NPM 6.0.0
Yarn 1.6.0
macOS High Sierra 10.13.4 (17E202)

Repro steps

Here's a link to our repo where the problem happens: https://github.com/Ninja-Squad/globe42.git

Repro steps:

• git clone https://github.com/Ninja-Squad/globe42.git
• cd globe42/frontend
• git checkout cli-test-regression-before
• yarn
• ng test --> all tests pass
• git checkout cli-test-regression-after: the only difference with the before branch is the versions of the CLI and the devkit, in package.json and yarn.lock
• yarn
• ng test --> one test fails, for a reason that I can't explain

The log given by the failure

HeadlessChrome 0.0.0 (Mac OS X 10.13.4) TasksPageComponent should navigate to other page when clicking page FAILED
	Expected spy navigate to have been called with [ [ '.' ], Object({ relativeTo: Object({ data: Observable({ _isScalar: true, _subscribe: Function, value: Object({ taskListType: 'todo', tasks: Object({ content: [ Object({ id: 0, description: 'Some description', title: 'Some title', category: Object, dueDate: '2017-08-01', status: 'DONE', totalSpentTimeInMinutes: 0, assignee: null, creator: Object, concernedPerson: null }), Object({ id: 1, description: 'Some description', title: 'Some title', category: Object, dueDate: '2017-08-01', status: 'DONE', totalSpentTimeInMinutes: 0, assignee: null, creator: Object, concernedPerson: null }), Object({ id: 2, description: 'Some description', title: 'Some title', category: Object, dueDate: '2017-08-01', status: 'DONE', totalSpentTimeInMinutes: 0, assignee: null, creator: Object, concernedPerson: null }) ], number: 0, size: 3, totalElements: 8, totalPages: 3 }) }) }), parent: Object({ snapshot: Object({ data: Object({  }) }) }) }), queryParams: Object({ page: '1' }) }) ] but it was never called.
	    at http://localhost:9876/_karma_webpack_/webpack:/src/app/tasks-page/tasks-page.component.spec.ts:143:31
	    at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.invoke node_modules/zone.js/dist/zone.js:388:1)
	    at AsyncTestZoneSpec.push../node_modules/zone.js/dist/zone-testing.js.AsyncTestZoneSpec.onInvoke node_modules/zone.js/dist/zone-testing.js:713:1)
	    at ProxyZoneSpec.push../node_modules/zone.js/dist/zone-testing.js.ProxyZoneSpec.onInvoke node_modules/zone.js/distHeadlessChrome 0.0.0 (Mac OS X 10.13.4) TasksPageComponent should navigate to other page when clicking page FAILED
	Expected spy navigate to have been called with [ [ '.' ], Object({ relativeTo: Object({ data: Observable({ _isScalar: true, _subscribe: Function, value: Object({ taskListType: 'todo', tasks: Object({ content: [ Object({ id: 0, description: 'Some description', title: 'Some title', category: Object, dueDate: '2017-08-01', status: 'DONE', totalSpentTimeInMinutes: 0, assignee: null, creator: Object, concernedPerson: null }), Object({ id: 1, description: 'Some description', title: 'Some title', category: Object, dueDate: '2017-08-01', status: 'DONE', totalSpentTimeInMinutes: 0, assignee: null, creator: Object, concernedPerson: null }), Object({ id: 2, description: 'Some description', title: 'Some title', category: Object, dueDate: '2017-08-01', status: 'DONE', totalSpentTimeInMinutes: 0, assignee: null, creator: Object, concernedPerson: null }) ], number: 0, size: 3, totalElements: 8, totalPages: 3 }) }) }), parent: Object({ snapshot: Object({ data: Object({  }) }) }) }), queryParams: Object({ page: '1' }) }) ] but it was never called.
	    at http://localhost:9876/_karma_webpack_/webpack:/src/app/tasks-page/tasks-page.component.spec.ts:143:31
	    at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.invoke node_modules/zone.js/dist/zone.js:388:1)
	    at AsyncTestZoneSpec.push../node_modules/zone.js/dist/zone-testing.js.AsyncTestZoneSpec.onInvoke node_modules/zone.js/dist/zone-testing.js:713:1)
	    at ProxyZoneSpec.push../node_modules/zone.js/dist/zone-testing.js.ProxyZoneSpec.onInvoke node_modules/zone.js/dist/zone-testing.js:285:1)
HeadlessChrome 0.0.0 (Mac OS X 10.13.4): Executed 394 of 465 (1 FAILED) (0 secs / 28.893 secs)
HeadlessChrome 0.0.0 (Mac OS X 10.13.4) TasksPageComponent should navigate to other page when clicking page FAILED
	Expected spy navigate to have been called with [ [ '.' ], Object({ relativeTo: Object({ data: Observable({ _isScalar: true, _subscribe: Function, value: Object({ taskListType: 'todo', tasks: Object({ content: [ Object({ id: 0, description: 'Some description', title: 'Some title', category: Object, dueDate: '2017-08-01', status: 'DONE', totalSpentTimeInMinutes: 0, assignee: null, creator: Object, concernedPerson: null }), Object({ id: 1, description: 'Some description', title: 'Some title', category: Object, dueDate: '2017-08-01', status: 'DONE', totalSpentTimeInMinutes: 0, assignee: null, creator: Object, concernedPerson: null }), Object({ id: 2, description: 'Some description', title: 'Some title', category: Object, dueDate: '2017-08-01', status: 'DONE', totalSpentTimeInMinutes: 0, assignee: null, creator: Object, concernedPerson: null }) ], number: 0, size: 3, totalElements: 8, totalPages: 3 }) }) }), parent: Object({ snapshot: Object({ data: Object({  }) }) }) }), queryParams: Object({ page: '1' }) }) ] but it was never called.
	    at http://localhost:9876/_karma_webpack_/webpack:/src/app/tasks-page/tasks-page.component.spec.ts:143:31
	    at ZoneDelegate.push../node_modules/zone.js/dist/zone.js.ZoneDelegate.invoke node_modules/zone.js/dist/zone.js:388:1)
	    at AsyncTestZoneSpec.push../node_modules/zone.js/dist/zone-testing.js.AsyncTestZoneSpec.onInvoke node_modules/zone.js/dist/zone-testing.js:713:1)
	    at ProxyZoneSpec.push../node_modules/zone.js/dist/zone-testing.js.ProxyZoneSpec.onInvoke node_modules/zone.js/distHeadlessChrome 0.0.0 (Mac OS X 10.13.4): Executed 465 of 465 (1 FAILED) (35.172 secs / 34.964 secs)

Desired functionality

The test seems correct, and has always passed with previous versions. Upgrading the CLI shouldn't make it fail.

Mention any other details that might be useful

[jnizet]

I've spent some time trying to figure out why the test fails, without success. Here are some observations, though:

The test consists, after some setup, in doing the following:

  it('should navigate to other page when clicking page', async(() => {
    const fixture = TestBed.createComponent(TasksPageComponent);
    fixture.detectChanges();

    const paginationFixture = fixture.debugElement.query(By.directive(NgbPagination));

    const router = TestBed.get(Router);
    spyOn(router, 'navigate');

    const page2Link = paginationFixture.nativeElement.querySelectorAll('a.page-link')[2];
    expect(page2Link.textContent).toContain('2');
    page2Link.click();

    fixture.detectChanges();
    fixture.whenStable().then(() => {
      expect(router.navigate).toHaveBeenCalledWith(['.'], {relativeTo: activatedRoute, queryParams: {page: '1'}});
    });
  }));

The click on page2Link is supposed to call the following method of the component:

  loadPage(viewPageNumber: number) {
    this.router.navigate(['.'], {relativeTo: this.route, queryParams: {page: (viewPageNumber - 1).toString()}});
  }

Observations:

• the test always passes with the CLI 6.0.7
• the test passes with the CLI 6.0.8 if I run it using fit or fdescribe, but fails when all the tests are run (which is already quite surprising)
• Adding logs to the loadPage method shows that it is being called by the test even when the test fails
• I added an expectation to check that the router obtained from the TestBed is the same as the router injected in the component (by making it public), and the expectation passes

[danwulff]

I'm also getting inconsistent results in unit tests with 6.0.8. This also revolves around navigation and navigation urls. Will try to get a min repo together when I have a chance.

Edit: couldn't create a min repo without risking spending too much time. Sorry.

[filipesilva]

@jnizet can you make a simple repro that shows this? It's hard to isolate the problem in the real project.

Also, can you try with @angular/cli@6.1.0-rc.3 and @angular-devkit/build-angular@0.7.0-rc.2?

[jnizet]

@filipesilva I've tried, for hours, but couldn't. Even the real test, in the project, passed when run alone (with fit), but failed when run with other tests. I've managed to identify that the loadPage() method was called after the callback passed to whenStable() (whatever the number of detectChanges/whenStable calls I could make. I've since switched to using fakeAsync to make the test pass, but I still don't have any explanation about the failure.

I'll try to see if the original test passes with the new CLI as you suggest, and will tell you about it (probably tomorrow)

[jnizet]

The issue is still there with @angular/cli@6.1.0-rc.3 and @angular-devkit/build-angular@0.7.0-rc.2