fix(ngcc): prevent including JavaScript sources outside of the package #37596

JoostK · 2020-06-15T22:34:45Z

See individual commits for details.

Fixes #37508

petebacondarwin

Couple of nits.
Nice and clean PR!

packages/compiler-cli/ngcc/src/packages/ngcc_compiler_host.ts

petebacondarwin · 2020-06-25T21:01:02Z

packages/compiler-cli/ngcc/test/helpers/utils.ts

@@ -68,7 +68,7 @@ export function makeTestBundleProgram(
  const rootDir = fs.dirname(entryPointPath);
  const options: ts.CompilerOptions =
      {allowJs: true, maxNodeModuleJsDepth: Infinity, checkJs: false, rootDir, rootDirs: [rootDir]};
-  const host = new NgccSourcesCompilerHost(fs, options, entryPointPath);
+  const host = new NgccSourcesCompilerHost(fs, options, rootDir);


I would rather we change the makeTestBundleProgram to accept an additional packagePath parameter, rather than relying upon the package being two directories up from the source file path.

This code has always made the assumption that path corresponds with a secondary entry-point, while that is not true in most cases. Cleaning up would likely be a large task, given that this function has ~480 callers. I could add an optional param here but that doesn't sounds like a proper way to go either.

packages/compiler-cli/ngcc/test/integration/ngcc_spec.ts

…rce files Changes `isWithinPackage` to take an `AbsoluteFsPath` instead of `ts.SourceFile`, to allow for an upcoming change to use it when no `ts.SourceFile` is available, but just a path.

When ngcc creates an entry-point program, the `allowJs` option is enabled in order to operate on the JavaScript source files of the entry-point. A side-effect of this approach is that external modules that don't ship declaration files will also have their JavaScript source files loaded into the program, as the `allowJs` flag allows for them to be imported. This may pose an issue in certain edge cases, where ngcc would inadvertently operate on these external modules. This can introduce all sorts of undesirable behavior and incompatibilities, e.g. the reflection host that is selected for the entry-point's format could be incompatible with that of the external module's JavaScript bundles. To avoid these kinds of issues, module resolution that would resolve to a JavaScript file located outside of the package will instead be rejected, as if the file would not exist. This would have been the behavior when `allowJs` is set to false, which is the case in typical Angular compilations. Fixes angular#37508

JoostK · 2020-06-29T18:46:35Z

merge-assistance: angular bot seems to have stalled, all checks should have passed.

#37596) When ngcc creates an entry-point program, the `allowJs` option is enabled in order to operate on the JavaScript source files of the entry-point. A side-effect of this approach is that external modules that don't ship declaration files will also have their JavaScript source files loaded into the program, as the `allowJs` flag allows for them to be imported. This may pose an issue in certain edge cases, where ngcc would inadvertently operate on these external modules. This can introduce all sorts of undesirable behavior and incompatibilities, e.g. the reflection host that is selected for the entry-point's format could be incompatible with that of the external module's JavaScript bundles. To avoid these kinds of issues, module resolution that would resolve to a JavaScript file located outside of the package will instead be rejected, as if the file would not exist. This would have been the behavior when `allowJs` is set to false, which is the case in typical Angular compilations. Fixes #37508 PR Close #37596

…rce files (#37596) Changes `isWithinPackage` to take an `AbsoluteFsPath` instead of `ts.SourceFile`, to allow for an upcoming change to use it when no `ts.SourceFile` is available, but just a path. PR Close #37596

#37596) When ngcc creates an entry-point program, the `allowJs` option is enabled in order to operate on the JavaScript source files of the entry-point. A side-effect of this approach is that external modules that don't ship declaration files will also have their JavaScript source files loaded into the program, as the `allowJs` flag allows for them to be imported. This may pose an issue in certain edge cases, where ngcc would inadvertently operate on these external modules. This can introduce all sorts of undesirable behavior and incompatibilities, e.g. the reflection host that is selected for the entry-point's format could be incompatible with that of the external module's JavaScript bundles. To avoid these kinds of issues, module resolution that would resolve to a JavaScript file located outside of the package will instead be rejected, as if the file would not exist. This would have been the behavior when `allowJs` is set to false, which is the case in typical Angular compilations. Fixes #37508 PR Close #37596

matheo · 2020-07-16T17:17:56Z

@JoostK help please!
We're upgrading from Angular v8 to v9
and our libraries are aliasing the exports with the company prefix.
So VendorService is exported as PrefixedVendorService.

I'm trying to upgrade a website to v9 keeping the Angular v8 libraries built with ng-packagr.
but Ivy is throwing this kind of exception while building the website for production:

ERROR in ./node_modules/@company/communication/__ivy_ngcc__/fesm2015/app.js 5011:315-335
"export 'VendorService' (imported as 'ɵngcc7') was not found in '@company/services'
    at HarmonyImportSpecifierDependency._getErrors (./node_modules/webpack/lib/dependencies/HarmonyImportSpecifierDependency.js:109:11)
    at HarmonyImportSpecifierDependency.getErrors (./node_modules/webpack/lib/dependencies/HarmonyImportSpecifierDependency.js:68:16)
    at Compilation.reportDependencyErrorsAndWarnings (./node_modules/webpack/lib/Compilation.js:1463:22)
    at hooks.finishModules.callAsync.err (./node_modules/webpack/lib/Compilation.js:1258:10)
    at AsyncSeriesHook.eval [as callAsync] (eval at create (./node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:33:1)
    at AsyncSeriesHook.lazyCompileHook (./node_modules/tapable/lib/Hook.js:154:20)
    at Compilation.finish (./node_modules/webpack/lib/Compilation.js:1253:28)
    at hooks.make.callAsync.err (./node_modules/webpack/lib/Compiler.js:672:17)
    at _done (eval at create (./node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:9:1)
    at _err1 (eval at create (./node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:36:22)
    at _addModuleChain (./node_modules/webpack/lib/Compilation.js:1185:12)
    at processModuleDependencies.err (./node_modules/webpack/lib/Compilation.js:1097:9)
    at process._tickCallback (internal/process/next_tick.js:61:11)
 @ ./src/app/applications/applications.module.ts
 @ ./src/app/app.module.ts
 @ ./src/main.ts
 @ multi ./src/main.ts

changing the target of the libraries from es5 to es2015 thrown a different error mentioning the missing generic of ModuleWithProviders. I used <any> and this exception came back.

I've also used this postinstall script with no joy:
ngcc --properties es2015 browser module main --first-only --create-ivy-entry-points

Do you have any clue to share with us? or we should disable Ivy in the meantime? :(

Angular CLI: 9.1.11
Node: 10.18.0
OS: linux x64

Angular: 9.1.12
... animations, common, compiler, compiler-cli, core, forms
... language-service, platform-browser, platform-browser-dynamic
... router
Ivy Workspace: Yes

Package                           Version
-----------------------------------------------------------
@angular-devkit/architect         0.901.11
@angular-devkit/build-angular     0.901.11
@angular-devkit/build-optimizer   0.901.11
@angular-devkit/build-webpack     0.901.11
@angular-devkit/core              9.1.11
@angular-devkit/schematics        9.1.11
@angular/cdk                      9.2.4
@angular/cli                      9.1.11
@ngtools/webpack                  9.1.11
@schematics/angular               9.1.11
@schematics/update                0.901.11
rxjs                              6.5.5
typescript                        3.8.3
webpack                           4.42.0

matheo · 2020-07-16T17:31:20Z

Reading #32902 I guess that something is happening with the definition files
because I see the aliases at node_modules/@company/services/index.d.ts
but Ivy keeps looking for the unprefixed names :(

JoostK · 2020-07-16T19:47:55Z

@matheo I can't tell from the top of my head, sorry. Aliased exports are generally supported I think. For some reason, compiling @company/communication introduced an import of the internal name VendorService from @company/services, whereas you mention this has only been exported as using a prefix. I briefly looked into how that could happen but I was unable to determine a potential path, so it's a bit hard offer any advice.

Regarding the ModuleWithProviders error, the generic type has only been made required in Angular 10, and your version report shows that you're using Angular 9. It could be that the Angular compiled itself required the generic type and produced an error, however ngcc should have added it automatically for libraries. Note that using any as the generic type arg is definitely an issue, as it must be the NgModule type that should be imported.

Without a reproduction I'm afraid I can't really give more accurate and actionable advice :-( We have not seen issues with private declarations together with ngcc in a very long time, so I'm really not sure what is wrong here.

If you are able to reproduce, do feel free to open a new issue. This PR does not seem relevant so this may not be the best place to discuss your problem ;-)

matheo · 2020-07-16T23:46:05Z

Thanks for the answer @JoostK
I used this PR as excuse because I wanted your opinion :)

I will trace the problem a bit and open a new issue to discuss this particular case.
Thanks again for your time!

angular-automatic-lock-bot · 2020-08-16T16:16:50Z

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}

…rce files (angular#37596) Changes `isWithinPackage` to take an `AbsoluteFsPath` instead of `ts.SourceFile`, to allow for an upcoming change to use it when no `ts.SourceFile` is available, but just a path. PR Close angular#37596

angular#37596) When ngcc creates an entry-point program, the `allowJs` option is enabled in order to operate on the JavaScript source files of the entry-point. A side-effect of this approach is that external modules that don't ship declaration files will also have their JavaScript source files loaded into the program, as the `allowJs` flag allows for them to be imported. This may pose an issue in certain edge cases, where ngcc would inadvertently operate on these external modules. This can introduce all sorts of undesirable behavior and incompatibilities, e.g. the reflection host that is selected for the entry-point's format could be incompatible with that of the external module's JavaScript bundles. To avoid these kinds of issues, module resolution that would resolve to a JavaScript file located outside of the package will instead be rejected, as if the file would not exist. This would have been the behavior when `allowJs` is set to false, which is the case in typical Angular compilations. Fixes angular#37508 PR Close angular#37596

JoostK added state: WIP comp: ngcc comp: compiler labels Jun 15, 2020

ngbot bot added this to the needsTriage milestone Jun 15, 2020

googlebot added the cla: yes label Jun 15, 2020

JoostK force-pushed the graceful-evaluation-errors branch from b00882b to d64e501 Jun 17, 2020

JoostK added a commit to JoostK/ngcc-validation that referenced this issue Jun 18, 2020

Test artifacts from angular/angular#37596

ff25b9d

JoostK force-pushed the graceful-evaluation-errors branch from d64e501 to 60cb313 Jun 18, 2020

This comment has been hidden.

Sign in to view

googlebot added cla: no and removed cla: yes labels Jun 18, 2020

JoostK force-pushed the graceful-evaluation-errors branch from 60cb313 to a159848 Jun 18, 2020

This comment has been hidden.

Sign in to view

googlebot added cla: yes and removed cla: no labels Jun 18, 2020

JoostK added a commit to JoostK/ngcc-validation that referenced this issue Jun 18, 2020

Test artifacts from angular/angular#37596

2d3711d

JoostK added a commit to JoostK/ngcc-validation that referenced this issue Jun 18, 2020

Test artifacts from angular/angular#37596

544e62a

JoostK force-pushed the graceful-evaluation-errors branch from a159848 to f479f83 Jun 23, 2020

JoostK added a commit to JoostK/ngcc-validation that referenced this issue Jun 23, 2020

Test artifacts from angular/angular#37596

3ab1e15

JoostK force-pushed the graceful-evaluation-errors branch from f479f83 to 036d740 Jun 25, 2020

JoostK changed the title ~~Graceful evaluation errors~~ fix(ngcc): prevent including JavaScript sources outside of the package Jun 25, 2020

JoostK added action: review target: patch and removed state: WIP labels Jun 25, 2020

JoostK marked this pull request as ready for review Jun 25, 2020

JoostK force-pushed the graceful-evaluation-errors branch from 036d740 to 153ef58 Jun 25, 2020

petebacondarwin approved these changes Jun 25, 2020

View changes

petebacondarwin added the action: cleanup label Jun 25, 2020

JoostK added 2 commits Jun 26, 2020

refactor(ngcc): let `isWithinPackage` operate on paths instead of sou…

455f1e4

…rce files Changes `isWithinPackage` to take an `AbsoluteFsPath` instead of `ts.SourceFile`, to allow for an upcoming change to use it when no `ts.SourceFile` is available, but just a path.

JoostK force-pushed the graceful-evaluation-errors branch from 153ef58 to 2b915f7 Jun 26, 2020

JoostK added action: merge action: merge-assistance and removed action: cleanup action: review labels Jun 26, 2020

AndrewKushnir closed this in 6b565ba Jun 29, 2020

angular-automatic-lock-bot bot locked and limited conversation to collaborators Aug 16, 2020

angular / angular Public

fix(ngcc): prevent including JavaScript sources outside of the package #37596

fix(ngcc): prevent including JavaScript sources outside of the package #37596

JoostK commented Jun 15, 2020 •

edited

This comment has been hidden.

This comment has been hidden.

petebacondarwin left a comment

petebacondarwin Jun 25, 2020

JoostK Jun 26, 2020

JoostK commented Jun 29, 2020

matheo commented Jul 16, 2020 •

edited

matheo commented Jul 16, 2020 •

edited

JoostK commented Jul 16, 2020 •

edited

matheo commented Jul 16, 2020

angular-automatic-lock-bot bot commented Aug 16, 2020

angular / angular Public

fix(ngcc): prevent including JavaScript sources outside of the package #37596

fix(ngcc): prevent including JavaScript sources outside of the package #37596

Conversation

JoostK commented Jun 15, 2020 • edited

This comment has been hidden.

This comment has been hidden.

petebacondarwin left a comment

petebacondarwin Jun 25, 2020

JoostK Jun 26, 2020

JoostK commented Jun 29, 2020

matheo commented Jul 16, 2020 • edited

matheo commented Jul 16, 2020 • edited

JoostK commented Jul 16, 2020 • edited

matheo commented Jul 16, 2020

angular-automatic-lock-bot bot commented Aug 16, 2020

JoostK commented Jun 15, 2020 •

edited

matheo commented Jul 16, 2020 •

edited

matheo commented Jul 16, 2020 •

edited

JoostK commented Jul 16, 2020 •

edited