bug(update): Cannot find module '@angular-devkit/core/src/workspace/json/reader'

[Splaktar]

Reproduction

Use StackBlitz to reproduce your issue: https://stackblitz.com/fork/components-issue

Steps to reproduce:

1. ng update @angular/core @angular/cli @angular/material --force

Expected Behavior

What behavior were you expecting to see?

• CDK Migrations run w/o exceptions.

Actual Behavior

What behavior did you actually see?

[error] Error: Cannot find module '@angular-devkit/core/src/workspace/json/reader'
Require stack:
- /Users/splaktar/Git/devintent/xlts/xlts.dev/node_modules/@angular/cdk/schematics/utils/project-tsconfig-paths.js
- /Users/splaktar/Git/devintent/xlts/xlts.dev/node_modules/@angular/cdk/schematics/ng-update/devkit-migration-rule.js
- /Users/splaktar/Git/devintent/xlts/xlts.dev/node_modules/@angular/cdk/schematics/ng-update/index.js
- /private/var/folders/mz/pjttw4vs5xs33pzstmq1h7140000gn/T/angular-cli-packages-JfwbdW/node_modules/@angular-devkit/schematics/tools/export-ref.js
- /private/var/folders/mz/pjttw4vs5xs33pzstmq1h7140000gn/T/angular-cli-packages-JfwbdW/node_modules/@angular-devkit/schematics/tools/index.js
- /private/var/folders/mz/pjttw4vs5xs33pzstmq1h7140000gn/T/angular-cli-packages-JfwbdW/node_modules/@angular/cli/utilities/json-schema.js
- /private/var/folders/mz/pjttw4vs5xs33pzstmq1h7140000gn/T/angular-cli-packages-JfwbdW/node_modules/@angular/cli/models/command-runner.js
- /private/var/folders/mz/pjttw4vs5xs33pzstmq1h7140000gn/T/angular-cli-packages-JfwbdW/node_modules/@angular/cli/lib/cli/index.js
- /private/var/folders/mz/pjttw4vs5xs33pzstmq1h7140000gn/T/angular-cli-packages-JfwbdW/node_modules/@angular/cli/lib/init.js
- /private/var/folders/mz/pjttw4vs5xs33pzstmq1h7140000gn/T/angular-cli-packages-JfwbdW/node_modules/@angular/cli/bin/ng
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:880:15)
    at Function.Module._load (internal/modules/cjs/loader.js:725:27)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/Users/splaktar/Git/devintent/xlts/xlts.dev/node_modules/@angular/cdk/schematics/utils/project-tsconfig-paths.js:21:18)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)
    at Module.require (internal/modules/cjs/loader.js:952:19)
    at require (internal/modules/cjs/helpers.js:88:18)
    at Object.<anonymous> (/Users/splaktar/Git/devintent/xlts/xlts.dev/node_modules/@angular/cdk/schematics/ng-update/devkit-migration-rule.js:22:34)
    at Module._compile (internal/modules/cjs/loader.js:1063:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
    at Module.load (internal/modules/cjs/loader.js:928:32)
    at Function.Module._load (internal/modules/cjs/loader.js:769:14)

Environment

• Browser(s): Chrome 86
• Operating System (e.g. Windows, macOS, Ubuntu): macOS 10.15.7

Before running ng update @angular/core @angular/cli @angular/material --force:

Angular CLI: 10.1.7
Node: 14.15.0
OS: darwin x64

Angular: 10.1.6
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router
Ivy Workspace: Yes

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1001.7
@angular-devkit/build-angular   0.1001.7
@angular-devkit/core            7.3.10
@angular-devkit/schematics      7.3.10
@angular/cdk                    10.2.7
@angular/cli                    10.1.7
@angular/fire                   6.0.5
@angular/material               10.2.7
@schematics/angular             10.1.7
@schematics/update              0.1001.7
ng-packagr                      10.1.2
rxjs                            6.6.3
typescript                      4.0.5

After running ng update @angular/core @angular/cli @angular/material --force:

Angular CLI: 11.0.1
Node: 14.15.0
OS: darwin x64

Angular: 11.0.0
... animations, cdk, common, compiler, compiler-cli, core, forms
... language-service, material, platform-browser
... platform-browser-dynamic, router
Ivy Workspace: Yes

Package                         Version
---------------------------------------------------------
@angular-devkit/architect       0.1100.1
@angular-devkit/build-angular   0.1100.1
@angular-devkit/core            7.3.10
@angular-devkit/schematics      7.3.10
@angular/cli                    11.0.1
@angular/fire                   6.0.5
@schematics/angular             11.0.1
@schematics/update              0.1100.1
ng-packagr                      11.0.2
rxjs                            6.6.3
typescript                      4.0.5

The CLI DevKit versions reported are v7 because of @schuchard/prettier and some others are using v8 or v9 versions of CLI DevKit:

npm ls @angular-devkit/core
xlts.dev@0.3.0 /Users/splaktar/Git/devintent/xlts/xlts.dev
├─┬ @angular-devkit/build-angular@0.1001.7
│ ├─┬ @angular-devkit/architect@0.1001.7
│ │ └── @angular-devkit/core@10.1.7 
│ ├─┬ @angular-devkit/build-webpack@0.1001.7
│ │ └── @angular-devkit/core@10.1.7 
│ ├── @angular-devkit/core@10.1.7 
│ └─┬ @ngtools/webpack@10.1.7
│   └── @angular-devkit/core@10.1.7 
├─┬ @angular/cli@10.1.7
│ ├── @angular-devkit/core@10.1.7 
│ ├─┬ @angular-devkit/schematics@10.1.7
│ │ └── @angular-devkit/core@10.1.7  deduped
│ ├─┬ @schematics/angular@10.1.7
│ │ ├── @angular-devkit/core@10.1.7 
│ │ └─┬ @angular-devkit/schematics@10.1.7
│ │   └── @angular-devkit/core@10.1.7  deduped
│ └─┬ @schematics/update@0.1001.7
│   ├── @angular-devkit/core@10.1.7 
│   └─┬ @angular-devkit/schematics@10.1.7
│     └── @angular-devkit/core@10.1.7  deduped
├─┬ @schuchard/prettier@3.1.0
│ ├── @angular-devkit/core@7.3.10 
│ └─┬ @angular-devkit/schematics@7.3.10
│   └── @angular-devkit/core@7.3.10  deduped
└─┬ @scullyio/init@1.0.1
  ├── @angular-devkit/core@9.1.12 
  ├─┬ @angular-devkit/schematics@9.1.12
  │ └── @angular-devkit/core@9.1.12  deduped
  ├─┬ @schematics/angular@9.1.12
  │ └── @angular-devkit/core@9.1.12  deduped
  └─┬ schematics-utilities@2.0.2
    ├── @angular-devkit/core@8.3.29 
    ├─┬ @angular-devkit/schematics@8.3.29
    │ └── @angular-devkit/core@8.3.29  deduped
    ├─┬ @schematics/angular@8.3.29
    │ └── @angular-devkit/core@8.3.29  deduped
    └─┬ @schematics/update@0.803.29
      └── @angular-devkit/core@8.3.29  deduped

It looks like the CDK is accessing the wrong version of the CLI DevKit SDK. I think that we had a similar problem in a previous release.

Command output:

$ ng update @angular/core @angular/cli @angular/material --allow-dirty --force
The installed local Angular CLI version is older than the latest stable version.
Installing a temporary version to perform the update.
Installing packages for tooling via npm.
Installed packages for tooling via npm.
Using package manager: 'npm'
Collecting installed dependencies...
Found 54 dependencies.
Fetching dependency metadata from registry...
                  Package "@angular-devkit/build-angular" has an incompatible peer dependency to "karma" (requires "~5.1.0", would install "5.2.3")
    Updating package.json with dependency @angular-devkit/build-angular @ "0.1100.1" (was "0.1001.7")...
    Updating package.json with dependency @angular/cli @ "11.0.1" (was "10.1.7")...
    Updating package.json with dependency @angular/compiler-cli @ "11.0.0" (was "10.1.6")...
    Updating package.json with dependency @angular/language-service @ "11.0.0" (was "10.1.6")...
    Updating package.json with dependency ng-packagr @ "11.0.2" (was "10.1.2")...
    Updating package.json with dependency @angular/animations @ "11.0.0" (was "10.1.6")...
    Updating package.json with dependency @angular/cdk @ "11.0.0" (was "10.2.7")...
    Updating package.json with dependency @angular/common @ "11.0.0" (was "10.1.6")...
    Updating package.json with dependency @angular/compiler @ "11.0.0" (was "10.1.6")...
    Updating package.json with dependency @angular/core @ "11.0.0" (was "10.1.6")...
    Updating package.json with dependency @angular/forms @ "11.0.0" (was "10.1.6")...
    Updating package.json with dependency @angular/material @ "11.0.0" (was "10.2.7")...
    Updating package.json with dependency @angular/platform-browser @ "11.0.0" (was "10.1.6")...
    Updating package.json with dependency @angular/platform-browser-dynamic @ "11.0.0" (was "10.1.6")...
    Updating package.json with dependency @angular/router @ "11.0.0" (was "10.1.6")...
  UPDATE package.json (3815 bytes)
✔ Packages installed successfully.
** Executing migrations of package '@angular/cli' **

❯ Replace deprecated library builder '@angular-devkit/build-ng-packagr'.
  Migration completed.

❯ Add 'declarationMap' compiler options for non production library builds.
  Migration completed.

❯ Remove deprecated options from 'angular.json' that are no longer present in v11.
  UPDATE angular.json (9080 bytes)
  Migration completed.

❯ Update workspace dependencies to match a new v11 project.
  UPDATE package.json (3816 bytes)
✔ Packages installed successfully.
  Migration completed.

An unhandled exception occurred: Cannot find module '@angular-devkit/core/src/workspace/json/reader'

[Splaktar]

PR #20745 removed the code that previously made our schematics devkit version-agnostic.

@alan-agius4 I think that we were under the impression "that the CLI would keep all the versions in line". Is this not the case or do you think that this is a bug in the CDK schematics?

[Splaktar]

OK, I verified that the Scully stuff isn't a problem. It's "@schuchard/prettier": "^3.1.0", that is causing the issue.

If I remove @schuchard/prettier, then re-run the ng update, everything works as expected.

[Splaktar]

I've opened schuchard/prettier-schematic#49 to see if those schematics can update their version of DevKit to make this no longer an immediate issue, but it still seems like our CDK schematics and/or the CLI should not break if the project has other schematics using an older DevKit version.

[alan-agius4]

This issue is quite similar to angular/angular-cli#18744 (comment) and share the same root causes.

The fundamental problem here is while @angular/core and @angular/material depends on @angular-devkit/schematics and @angular-devkit/core these are not listed as dependencies. Therefore in some cases when having 3rd party dependencies that install older versions of these packages one might end in a module layout which will break the mentioned schematics.

As mentioned previously it's not desired to have these as direct dependencies of @angular/core and @angular/material but that would be the "proper" fix to make sure that the proper module structure is laid out when running npm install / yarn install.

This is a known issue which is caused by the decision of not have explicit dependencies in a library and unfortunately, at this point there is not much we can do without a re-design of how ng update works.

[Splaktar]

That makes a lot of sense. If we have schematics in the library that depend on those packages, then they should be dependencies.

Should we at least add @angular-devkit/schematics and @angular-devkit/core to our peerDependencies (possibly as optional) so that developers would at least get a warning that these need to be installed?

I don't see how it's viable to not have those packages listed as some kind of dependency if our schematics depend on them. I guess an alternative is to break all of our schematics out into a separate package that has the correct dependencies.

[alan-agius4]

Should we at least add @angular-devkit/schematics and @angular-devkit/core to our peerDependencies so that developers would at least get a warning that these need to be installed?

Adding them as peerDependency will require this packages to be installed and listed in the users package.json, which is not ideal. Users shouldn’t be exposed to the packages unless they are developing schematics themselves. Also with NPM 7, unmet peerDepedencies get installed by default and unmeet peer deps are now hard error https://github.com/npm/rfcs/blob/latest/implemented/0025-install-peer-deps.md

I guess an alternative is to break all of our schematics out into a separate package that has the correct dependencies.

That can work, although I don’t think it should be a single package for all the migrations. I think it should be still separated by repo. I’d also assume that this package will be installed by the schematic when the migration runs.

Note: all the above will only be needed until a new version ng-update is available.

[Splaktar]

Users shouldn’t be exposed to the packages unless they are developing schematics themselves.

That makes sense. Now I get why it's better to make them direct dependencies.

I don’t think it should be a single package for all the migrations. I think it should be still separated by repo.

Agreed.

[telb99]

Still an issue...

ng update @angular/core @angular/cli --create-commits --force
The installed local Angular CLI version is older than the latest stable version.
Installing a temporary version to perform the update.
Installing packages for tooling via npm.
Installed packages for tooling via npm.
Using package manager: 'npm'
Collecting installed dependencies...
Found 50 dependencies.
Fetching dependency metadata from registry...
Package "@angular-devkit/build-angular" has an incompatible peer dependency to "karma" (requires "~5.1.0", would install "5.2.3")
Package "@ag-grid-community/angular" has an incompatible peer dependency to "@angular/common" (requires "^9.0.0" (extended), would install "11.0.8").
Package "@ag-grid-community/angular" has an incompatible peer dependency to "@angular/core" (requires "^9.0.0" (extended), would install "11.0.8").
Updating package.json with dependency @angular-devkit/build-angular @ "0.1100.6" (was "0.1002.1")...
Updating package.json with dependency @angular/cli @ "11.0.6" (was "10.2.1")...
Updating package.json with dependency @angular/compiler-cli @ "11.0.8" (was "10.2.4")...
Updating package.json with dependency @angular/language-service @ "11.0.8" (was "10.2.4")...
Updating package.json with dependency ng-packagr @ "11.0.3" (was "10.1.2")...
Updating package.json with dependency @angular/animations @ "11.0.8" (was "10.2.4")...
Updating package.json with dependency @angular/common @ "11.0.8" (was "10.2.4")...
Updating package.json with dependency @angular/compiler @ "11.0.8" (was "10.2.4")...
Updating package.json with dependency @angular/core @ "11.0.8" (was "10.2.4")...
Updating package.json with dependency @angular/forms @ "11.0.8" (was "10.2.4")...
Updating package.json with dependency @angular/platform-browser @ "11.0.8" (was "10.2.4")...
Updating package.json with dependency @angular/platform-browser-dynamic @ "11.0.8" (was "10.2.4")...
Updating package.json with dependency @angular/router @ "11.0.8" (was "10.2.4")...
UPDATE package.json (2802 bytes)
v Packages installed successfully.
Committed migration step (64fa6aadb): Angular CLI update for packages - @angular/core@latest, @angular/cli@latest.
** Executing migrations of package '@angular/cli' **

Replace deprecated library builder '@angular-devkit/build-ng-packagr'.
UPDATE angular.json (5810 bytes)
UPDATE package.json (2750 bytes)
Migration completed.
Committed migration step (25f741825): @angular/cli migration - replace-ng-packagr-builder.

Add 'declarationMap' compiler options for non production library builds.
UPDATE projects/ng-xlr8-toolkit/tsconfig.lib.json (563 bytes)
Migration completed.
Committed migration step (cae2f1865): @angular/cli migration - add-declaration-map-compiler-option.

Remove deprecated options from 'angular.json' that are no longer present in v11.
UPDATE angular.json (5797 bytes)
Migration completed.
Committed migration step (0fe985684): @angular/cli migration - update-angular-config-v11.

Update workspace dependencies to match a new v11 project.
UPDATE package.json (2749 bytes)
v Packages installed successfully.
Migration completed.
Committed migration step (0f90296c2): @angular/cli migration - update-workspace-dependencies-v11.

An unhandled exception occurred: Cannot find module '@angular-devkit/core'
Require stack:

• workspace\node_modules@angular\core\schematics\utils\project_tsconfig_paths.js
• workspace\node_modules@angular\core\schematics\migrations\missing-injectable\index.js
• {AppData}\Local\Temp\1\angular-cli-packages-ilHFVK\node_modules@angular\cli\node_modules@angular-devkit\schematics\tools\export-ref.js
• {AppData}\Local\Temp\1\angular-cli-packages-ilHFVK\node_modules@angular\cli\node_modules@angular-devkit\schematics\tools\index.js
• {AppData}\Local\Temp\1\angular-cli-packages-ilHFVK\node_modules@angular\cli\utilities\json-schema.js
• {AppData}\Local\Temp\1\angular-cli-packages-ilHFVK\node_modules@angular\cli\models\command-runner.js
• {AppData}\Local\Temp\1\angular-cli-packages-ilHFVK\node_modules@angular\cli\lib\cli\index.js
• {AppData}\Local\Temp\1\angular-cli-packages-ilHFVK\node_modules@angular\cli\lib\init.js
• {AppData}\Local\Temp\1\angular-cli-packages-ilHFVK\node_modules@angular\cli\bin\ng
  See "{AppData}\Local\Temp\1\ng-HAUUPW\angular-errors.log" for further details.