New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A Proposal For Module Resolution #50152
Comments
I think a very important question for this is: With what default conditions (configurable?) and what behavior for the import/require conditions? Each bundler runs different condition sets from what I know - most have a bundler-specific one, in addition to often using a |
|
Also, an important concern for declaration emit is cross-resolution mode compatibility. |
|
Also ref #29353 where I did what I thought would be some minimal emit changes to create a mode about as flexible as bundlers were at the time. That didn't include any resolution changes :) |
This is a problem for us and DT just about as much as it’s a problem for actual runtimes and bundlers consuming implementation packages from npm. What’s in DT should reflect what’s in the implementation package on npm. If the implementation package writes imports that only resolve under bizarre resolvers of the future, it will make sense for the DT package to do the same; it will make sense for users to have broken types under |
Just to add onto this (I'm sure this is known from the research put into this proposal, so forgive me if it's already been considered), esbuild has conditions totally configurable (https://esbuild.github.io/api/#conditions, https://esbuild.github.io/api/#main-fields); its behavior was almost changed in the last release which would have changed its defaults too in some scenarios to avoid dual-package hazards (evanw/esbuild#2417), since bundlers can sort of just "import" anything however they want regardless of the syntax (since they control the runtime import behavior internally). |
|
Seems like loaders for Node.js such as ts-node and tsx, or even a future version of Node.js, would possibly want to allow Would this be as simple as taking I do have some concerns with the approach of enabling |
I'm curious how this will play out with projects that interpret TS directly; I seem to recall one popular project actually requiring that you didn't set |
|
For anything actually running TS in |
Maybe to dovetail with #50133, we should have a |
|
@jakebailey I believe you’re thinking of ts-loader, but only in “full mode” as opposed to @DanielRosenwasser |
Not my last team... |
|
I feel confident that either it will not matter at all or that the combined will and know-how of me and @johnnyreilly will be up to the task. |
|
As a library author, one thing which would be nice is to allow moduleResolution to be an array, which tells typescript that imports must be valid in all of the listed types. So this would be even more restrictive, as library authors we need to write imports in a way that all the module resolutions work and typescript could help by giving an error. So we could specify There wouldn't really need to be any extra special logic in typescript, if moduleResolution is an array typescript would just process each import twice and error if they are not all equal. |
|
I guess its still time to mention TypeScript Should let the Host System do Specifier Lookup let me explain In Our Case this is TypeScript it self i my self Authored a Lot of Engines so Host Systems that Embedded ECMAScript my Proposal is finally Merge Rollup or at last its algo under module Resolution rollup into typescript Rollup is a loader / bundler and a universal ModuleSystem all in One. All Existing Host Engines do offer Ways to Manipulate Specifier Resolution
Advancedrollup uses Acorn as AST which is more flexible i suggest changing the Typescript AST with the Acorn one for better Interop in general while there exist Shims for both AST's to work with each other. the missing parts could get created as Acorn plugin More Big ProposalDo not Integrate a Plugin system for specifier lookup kick it all out and go back to only support moduleResolution Classic also supporting a lookup on package-manager level would be guess able. as mentioned the lockfiles do container specifier resolution at last for the complet module system of the whole project. I guess Classic + Files containing References or .d.ts only exports is the right way to go. This way we have the resolved parts like with a package-lock.json. |
|
Hi @andrewbranch this really interesting proposal, we have a recurring meeting at TC39 focusing on Tools with folks from(Deno, Parcel, etc..). It would be nice to bring this to the discussion. Send me a DM or fill the form in case you want to participate. :) |
|
about DTS resolution, can we emit code like this to let all declaration brings their own resolution mode instead of inheriting from the project? |
|
Thanks for taking the time to study this deeply. Module resolution is fundamental to the entire web tooling stack so getting it right is valuable and appreciated. Initial ReactionWe know this is a tough problem space from experience. Inside Bloomberg we have our own runtime environment (everyone seems to be doing it these days) with its own custom resolution semantics. As surmised in Background, just like all the other non-Node runtimes we chose to use Today we handle resolution of bare-specifiers using Explicitness as a Guiding PrincipleThe overall direction of this proposal is to permit explicitness in source code as a way of ensuring correctness. I think this is our North Star and we can use it to resolve some of the consequential issues. Explicitness matters most obviously in the runtime. As the web platform evolves to allow more file formats to be directly imported (JS, CSS, JSON, Wasm, et al) extensionless imports become a hazard. It's hard to know what Bundlers answer this question by implementing priority order. So the preference for our codebases is to avoid betting on a specific bundler's convention and instead sidestep this hazard by ensuring that relative specifiers in source code always use the full source code filename explicitly including its extension. Importing Declaration Files
This is an appreciated step forwards towards permitting and encouraging explicitness Low priority suggestion: For relative specifiers in Emitting relative module specifiers ending in
|
@robpalme Just so I'm clear -- your runtime is capable of handling a situation where there are no // a.js
import * as b from "./b.ts";
// b.js
export { }How does it know if there was ever a Edit: Fixed a critical typo in this example ( |
|
@RyanCavanaugh I'm describing a different case. Meaning if there is a |
|
@robpalme did you consider using references the tripple slash once to reference files from files? i do that today and it is a win for me. Google tripple slash typescript If you are not aware of that. for bundling i simply translate the tripple slash references to imports |
|
@robpalme and I discussed the emit vs The nuance here is that we would still intend to block |
|
@RyanCavanaugh there is one thing my thing that implements that i have some engines started with integrated Proposal from TC39 and @DanielRosenwasser and other People that i know that are like minded. i for example can take .js and .ts files while .ts files will not work if they use special features like enum but as long as they only use type annotations they will run my.js import { helloWorld } from './hello-world.ts';
const { log } = console;
log(helloWorld); // => 'Hello World' hello-world.ts export const helloWorld: string = 'Hello World'In the next JDK Release there will be a Engine Parameter for the JS Interop that enables that for testing. And on the long run i guess more engines will adopt that as it is a clever thing and solves typescripts biggest issue that you can not code something in .ts files that runs as it is. with that proposal it can run as is as long as it does not use Typescript Superset like enum. In none NodeJS Based Engines there is no Confusion what import returns it always returns ESModules no matter what the extension says no diffrent behavior in general the specifier behavior is total up to the host and out of the ECMAScript Engine. |
|
Frank, your linked TC39 proposal doesn't seem to set a convention for file extensions. It seems like a really bad idea to have one file extension that could mean both:
I would hope that we come up with a better convention for naming typed Javascript files. (Is |
|
@thw0rted mope you did not fully understand let me explain there is a ECMAScript Standard it contains 1 module system this module System is Defined in a Specifier Agnostic Way. In ECMAScript / JavaScript exists only ECMAScript and JavaScript i did not define that this is how it is. NodeJS Implemented a own loader and module system. @thw0rted there is no need for a extra extension ECMAScript calls everything in the string that you use to import as "Specifier" the Import Specifier needs to get resolved and loaded by the Host Environment (The one that embedded the JS Engine v8) so we could make a convention for directory names 👍🏻 that would be more flexible the host can simply use the correct loaded based on the indicator in the path |
|
I think I see what you're saying. If we look at support for the TC39 erasable type specification proposal the same way as any other new ES feature, like say the dynamic In that case, though, |
|
@thw0rted deno also imports ts with .ts specifiers there it is wide used and standard |
|
OK, so, TS is a special case, in that Deno can have a JS entry point that imports from a TS file (with a That still doesn't change the fact that ES20XX Totally Vanilla JS That Happens To Support Some Type Declarations, should carry a ETA: I just realized, this answers @RyanCavanaugh 's comment from a while back about "it's unreasonable to think that a runtime could see a |
My comment above had a critical typo in it. What doesn't happen (to my understanding) is that Deno sees an import of Resolution to extant files on disk (or files which will eventually exist, if we're talking pre-runtime) is, I think at this point, uncontroversial. Resolution from build outputs back to build inputs is the thing that I am asserting doesn't exist (and shouldn't). |
|
lets all vote and push Tc39 type annotations and drop the .ts exstension and call it a day. |
|
One minor note; internally we’ve been kicking around |
|
Sorry for the late post but i got a good idea including aux resource import Maps as source of truth for resolve. And ECMAScript got a standard for that kind of Build/Runtime Maps. Typescript should adopt the importMaps standard as any other ECMAScript Engine Implementer would do it TypeScript can be rated as Language (Runtime Distribution) that Embeddeds the ECMAScript Engine. So its Role is Host so it should use importMaps no matter what the internals are to create that. The current handling is more like a IDL Interface Definition Language not like a Runtime Coding Language. I guess Handling Typescript out of General Coding Language view makes more sense then handling it as configurable IDL |
Conventions don't change! We're all still manually rendering script tags and using global scope for imports. Right? |
andrewbranch commentedAug 2, 2022
Background
When a user writes a module specifier (the string literal after
fromin an import declaration) in a TypeScript file, how should the compiler resolve that string to a file on disk to be included in type checking? Because TypeScript never rewrites module specifiers in its JavaScript emit, the only possible answer is that it should mirror whatever resolution behavior the code’s intended runtime module resolver has. I’m using “runtime module resolver” to mean the system whose module resolution behavior has observable effects at runtime: it may be a component of the runtime itself, as in Node, or it may be a bundler that consumes the module specifiers to produce one or more script files. (The “runtime” distinction is made to exclude analysis tools like linters, which may perform module resolution without having any impact on runtime behavior.) TypeScript’s way of handling this has been to say that the user must indicate what their code’s runtime module resolver is via themoduleResolutioncompiler option so the compiler can mirror it.As little as five years ago, there were only two places JavaScript could run that were worth mentioning: in Node as CommonJS modules, and in the browser as scripts. For the former, TypeScript had
--moduleResolution node. (The latter needed nomoduleResolutionmode, though you can argue some sort ofnonevalue would have been appropriate.) However, bundlers like Webpack were widely used and were themselves module resolvers, perhaps deserving their ownmoduleResolutionsetting according to TypeScript’s philosophy. But demand for bundler-specific module resolution was essentially nonexistent, because bundlers mostly just copied Node’s module resolution algorithm, so users were able to get by with--moduleResolution node.(Note: as of this writing,
--moduleResolution node16and--moduleResolution nodenextare identical in TypeScript. The latter is intended to be updated as Node changes. For brevity, I usenode16in this writing, but both are equally applicable everywhere.)Over the next few years, though, the landscape changed. Browsers adopted ESM as a natively supported format, and Node added ESM support alongside CJS, with a complex interop system and new features like package.json
exports. A new wave of bundlers and runtimes emerged, and many adopted some of the features that Node introduced. But this time, none was similar enough to Node to piggyback on TypeScript’s--moduleResolution node16option without users noticing problems.Today
In this new landscape, users have been trying both
nodeandnode16with bundlers and browsers and hitting walls, which I will explore in some detail. In brief, the JavaScript ecosystem is in a phase where we cannot hope to provide a dedicatedmoduleResolutionmode for every runtime and bundler. At the same time, we have resisted allowing resolver plugins for many reasons:This philosophy has brought TypeScript to a point where we have avoided some significant pitfalls, but have essentially no support for module resolvers that are not Node. To make the issues explicit, let’s examine some hypothetical case studies.
Bundling with Webpack, esbuild, or Vite
These bundlers use a Node-CJS-like resolution algorithm and support package.json
exports. If the user chooses--moduleResolution node, any of their dependencies that modify their export structure via package.jsonexportswill be misrepresented by TypeScript—imports from that package may not resolve, they may resolve to incorrect files, and they will receive incorrect auto-imports and path completions. If the user chooses--moduleResolution node16, TypeScript will resolve their imports against dependencies’ package.jsonexports, but the conditions it uses in the lookup may be wrong: bundlers always set theimportcondition for imports and therequiredefinition for require calls, but TypeScript believes that import declarations in files that have not been explicitly scoped as ESM will be transpiled intorequirecalls, so it looks up these imports with therequirecondition, which could lead to incorrect resolutions. Moreover, in these files, TypeScript will prohibit imports (because it think they are actuallyrequires) of ESM-format files. The bundler has no such restriction, as its CJS/ESM divide is purely syntactic. (This is a slight oversimplification and the three bundlers mentioned behave slightly differently, but the simplification is good enough for describing the user experience.) If the user tries to get around this by scoping all of their files as ESM by setting"type": "module"in their own package.json, TypeScript will impose Node’s much stricter ESM resolution algorithm on those files, disabling index-file resolution and extensionless lookups—in fact, the extension the user has to write is.js, which will be nonsensical for the context, where the runtime module resolver (the bundler) only ever sees.tsfiles. (Vite and esbuild tolerate this extension mismatch out of the box; Webpack has historically required a plugin but just added a config setting for it.) This configuration satisfies both the bundler and TypeScript, but at a high DX cost for the user—TypeScript imposes rules on resolution that are wholly unnecessary for the bundler.Running in Bun
The situation is exactly the same as the above, since Bun’s module resolver is a port of esbuild’s, and it consumes TS files directly.
Bundling with Parcel or Browserify
These bundlers do not (yet) support package.json
exports, so--moduleResolution nodeis still a reasonably good fit.Writing ESM for the browser
Every module resolution mode except
classicperforms node_modules resolution, which does not happen in the browser.classicperforms index-file and extensionless lookups, which does not happen in the browser. The closest the user can get is probably to usenode16such that index-file and extensionless lookups are disabled, but they have to take care to avoid node_modules lookups and importing CommonJS dependencies.Writing ESM for Node, browser, or Deno
We have heard a few arguments recently about the ability to write code targeting multiple runtimes, mostly in the form of “if you let me write my imports with
.tsextensions and emit them as.jsextensions, my input files will work in Deno and my output files will work in Node” which is not generally true. However, it is true that Node, the browser, and Deno have a small amount of overlap in resolution behavior such that it is possible to write ES modules in JS and publish them both to npm and to a CDN where they can be consumed by browsers or Deno. A user trying to do this today faces the same situation as the case above, since the overlap between these systems is just relative URL imports including extensions: there is no mode restrictive enough to avoid writing imports that will work in Node but not in Deno or the browser. (Note that targeting a single bundler which produces a separate output for each target runtime is, for now, a better approach for multi-platform JS authoring.)Proposal
Existing module resolution modes (with the exception of
classic, whose existence is still a mystery to me) have intended to target one specific runtime and have been named for that runtime—node(v11 and before),node16, andnodenext—and the resolution features they entail are non-configurable implementation details. To move forward, I suggest a strategy of composition: expose some lower-level modes that can be combined with additional options to build up modes that are suitable for a variety of runtime resolvers. If the ecosystem converges on combinations of these settings, we can encapsulate them in a named mode. To start this process, I propose the following reorganization and expansion of options (all names subject to bikeshedding):Module Resolution Modes
nodeas a low-level mode calledconventional, with a slight modification necessary to support.tsextension resolution undernoEmit, and defaultingesModuleInteropto true. (The name, which I am more than happy to change, is a reference to the fact that most runtimes and bundlers have copied node_modules resolution, extensionless lookups, and special index-file handling from Node to the point where users no longer think of these features as specific to Node. Since we now havenode16which is highly Node-specific, the goal is to create a situation where the only people who should choose amoduleResolutionoption named after Node are people who are actually using Node.)nodeas a composition ofconventionaland an internal-only option that undoes the modifications mentioned in (1) to preserve backward compatibility. Also, deprecate the namenodein favor ofnode-legacyto encourage users of modern Node to consider migrating tonode16, and to encourage users of other runtimes and bundlers to consider migrating toconventional. Today’snodeis only accurate to Node v11 and earlier, so we need to start guiding people away from it at some point.minimalwhich resolves only relative module specifiers including file extensions, and attempts to parse all files as ESM. This can be used as a base for browser module resolution.classic,node16, andnodenextas they are.Module Resolution Options
exportsinconventionaland add conditions to the resolver. (May also apply to other modes that do node_modules resolution, i.e. everything butclassicandminimal.)That’s it for now—in the future, import maps, HTTP imports, and other features adopted by more than one runtime resolver should be exposed as options. When/if we have support for import maps and HTTP imports specifically, we should consider creating a mode named
browserthat is a composition ofminimaland those options defaulted to true.Resolution of relative module specifiers ending in
.tsBoth
minimalandconventional(but notnode-legacy) will support resolution to.tsfiles by specifying a.tsextension in the module specifier. This will be an error, as it is today, unlessnoEmitis enabled. This allows users who are bundling or directly running their TypeScript source to write relative module specifiers with the extension that their runtime module resolver will actually see, which has always been the underlying goal of telling users to write.jsextensions when their runtime resolver will operate on the emitted JS code. Additionally, in these modes, I suggest that it be legal to write animport typeof a module specifier ending in.d.ts.This cannot be supported in today’s
nodeornode16in a fully backward-compatible way. In these modes, an import of"./foo.ts"will resolve tofoo.ts.jsorfoo.ts.d.tsin the same directory even iffoo.tsis also present; unsupported extensions are not probed for existence before moving on to fallbacks. This amounts to a bug innodeandnode16. I have proposed to leave the bug in place fornode-legacyto preserve backward compatibility, but it may be reasonable to try fixing it everywhere and listen for feedback.It should be noted that
compositeprojects may not disable emit if they are referenced by another project. It may be possible to relax thenoEmitrestriction toemitDeclarationOnly. The primary challenge here is a portability concern: oldermoduleResolutionmodes will not be able to resolve the.ts-suffixed specifiers in those declaration files. I think it’s worth fixingnode16to support this; they would receive the bug fix described above so that they can always resolve.ts-suffixed specifiers, but would continue to issue a checker error. That way, we can safely silence the error in declaration files for better portability, and projects that need to use.ts-suffixed imports could becompositeproject references. Fixingnode16in this way would also prepare us for the possibility of Node running directly on TS files, transpiling in-memory like ts-node, an idea that has been gaining traction with Node maintainers recently.However, I think much of the demand we’ve heard so far for being able to use
.ts-suffixed module specifiers has been misplaced. Users who tried to usenode16with a bundler may have been prompted to add a.jsextension to a module specifier and thought that adding a.tsextension makes more sense, when in actuality they can continue to use extensionless imports in a mode likeconventional. Others demand.ts-suffixed imports in combination with module specifier rewriting because they believe that will let them write input code that will run natively in Deno, while tsc’s output will run natively in Node. This is out of scope; the way to write once and ship to multiple environments is to target a bundler and produce multiple bundles. Consequently, I think there are very few users who need to write.ts-suffixed imports (especially in a world withconventional), but they are unobjectionable innoEmitand easy to implement. The feature is not core to this proposal, but I believe it would be a mistake to create new module resolution modes without at least fixing the aforementioned bug to carve out the possibility of.ts-suffixed imports resolving in the future.Notes on
conventionalThis proposal does not allow for a perfect mapping of TypeScript’s resolution behavior onto every bundler, but I think it covers most cases, or what I will call all reasonable cases. If we wanted to be a bit prescriptive, I would be tempted to prohibit
.ctsand.cjsfiles, disallowimport m = require(...)syntax in TypeScript files, and disable resolution ofrequirecalls in JS files. Some of the newer bundlers are explicitly ESM-only, ignoring or prohibitingrequirecalls in user code and converting library dependencies from CJS to ESM. No bundler I tested had separate CJS and ESM resolution algorithms, with the exception of settingimportvs.requirein the resolver conditions when looking up package.jsonexports. There seems to be little reason to allow explicitly CJS constructs in implementation files in this mode (while CJS constructs in dependency declaration files obviously need to be consumable). As in TS files today, users will still be free to writerequirecalls, but they will not have special resolution behavior.Unanswered questions
--moduleResolution node, which is the default for--module commonjs. Consequently, many users are usingnodewithout realizing it. This raises the question of what defaults we should have in the future, whatmodulesettings should be allowed with thesemoduleResolutionmodes, and more broadly, how to guide users into selecting the correct settings for their project.extendsand encourage bundlers to publish (or publish ourselves under@typescript) tsconfig bases that reflect the resolution behaviors supported by these bundlers out-of-the-box. That way, an esbuild user could write.ts-suffixed resolution be automatically allowed based onnoEmit, or should it be gated behind another flag, or should the capability be preserved for the future but not enabled yet? It makes sense to me thatnoEmitshould enable it, because if you’re writing modules but not emitting, it stands to reason that another tool is going to consume the TS modules that you wrote. @DanielRosenwasser raised the idea that this may cannibalize project references usage, which requires declarations to be emitted and can help speed up type checking when splitting large codebases. More thought needs to be put into how.tsimports would work with declaration emit.node, breaking backward compatibility? The first time I considered this, I thought this would be an untenable breaking change, because people rely on this behavior to write declarations for files with unsupported extensions, e.g.import styles from "./styles.css"would resolve tostyles.css.d.tsbecause it thinks that’s analogous tostyles.css.js, notstyles.css. However, @weswigham has proposed a general solution for this at Proposal: Enable declaration files for non-js-extensioned files #50133. Taking some form of that proposal may be key to fixing this “bug” in any resolution mode without effectively losing a feature.Related: #37582, #49083, #46452, #46334, and probably a dozen others
The text was updated successfully, but these errors were encountered: