Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Bundling TS module type definitions #4433

Open
weswigham opened this issue Aug 25, 2015 · 55 comments
Open

Proposal: Bundling TS module type definitions #4433

weswigham opened this issue Aug 25, 2015 · 55 comments
Labels
In Discussion Suggestion

Comments

@weswigham
Copy link

@weswigham weswigham commented Aug 25, 2015

Relates to #3159, #4068, #2568, this branch, and this tool.

Goals

  • Bundle declarations for TS projects to allow a library to be consumed with a single TS file, despite being many modules internally. Because of complicated internal module dependencies which would rather not be exposed to the consumer, this should flatten the exported module types as best as it can. (Ideally, completely.)

Proposal

When all of --module, --out, and --declarations are specified, the TS compiler should emit a single amalgamated .d.ts (alongside its single output js file). This .d.ts should be flattened compared to a concatenated .d.ts file. It should report collisions caused by scoping issues and import aliasing when flattening declarations into a single declare module. It should respect access modifiers when generating the DTS (only exporting things explicitly exported and types marked as public).

For example, given the following set of sources:
tsconfig.json:

{
  "compilerOptions": {
    "module": "commonjs",
    "declarations": true,
    "out": "mylib.js"
  }
}

a.ts:

export * from './b';
export * from './c';

b.ts:

export interface Foo {}

export class Bar {
    constructor() {
        console.log('');
    }

    do(): Foo { throw new Error('Not implemented.'); }
}

c.ts:

export class Baz {}

should create the .d.ts:
mylib.d.ts:

declare module "mylib" {
  export interface Foo {}

  export class Bar {
    constructor()
    do(): Foo
  }

  export class Baz {}
}

rather than:
mylib.d.ts:

declare module "mylib/a" {
  export * from "mylib/b";
  export * from "mylib/c";
}
declare module "mylib/b" {
  export interface Foo {}

  export class Bar {
    constructor()
    do(): Foo
  }
}
declare module "mylib/c" {
  export class Baz {}
}
declare module "mylib" {
  export * from "mylib/a";
}

and should report a semantic error when the following is done:
a.ts:

export * from './b';
export {Bar as Foo} from './b';
export * from './c';

as there will be multiple members named Foo (an interface and a class), since b.ts has exported interface Foo.

We should also have a semantic error when the following is changed from the original:
If we change c.ts:

export class Baz {}
export interface Foo {}

it should be an error in a.ts (since it's blanket exporting b and c), and the error should suggest to alias either c.ts's Foo or b.ts's Foo (or both) when reexporting them in a.

Internally, when flattening this aliasing becomes important - we need to track usages of the two original Foo's across the generated .d.ts and rename it to the alias created when it is reexported.

Unfortunately, to maintain ES6 compatability, while we can warn about this behavior with classes (since it's possible that a developer is unaware they're overriding a prior export), we still need to support it (or do we? The spec leads me to believe that attempting to export multiple members with the same name - even via export * - is an early syntax error). So it would be nice to have a compiler flag to mark the same kind of thing with classes (or namespaces) as an error, but also do the following by default:

We can do automatic name collision resolution, but that can result in unpredictable (or convention-based) public member names... but it must be done, I suppose. We could ignore reexported types since it's appropriate to do so in ES6 (following export * declarations can override previously defined members? maybe? system works this way at present - but that may just be system relying on transpiler implementers to maintain ES6 semantics), then we would need to create "shadowed" types at the appropriate level in the .d.ts - types whose original public access are overridden by later exports but whose types are still required to describe public function argument or return types. Naming these "shadowed" types could be difficult, but given that they only exist for type information and not for access information, a common (re)naming convention could be a desirable solution. Something akin to <typename>_n when n is the shadowed type number for that type, and renaming the shadowed type name to something else (<typename>__n and so on so long as the name still exists) if that collides with another exported type. Classes used in this way are rewritten to interfaces in the .d.ts, since a constructor function likely isn't accessible for a shadowed class (at least not at its generated exported type name).

Any feedback? There's a few alternatives to what I've suggested here, which is possibly the most conservative approach in terms of ability to error early but supporting ES6 semantics best. It's possible to silently ignore interface name collisions and rename those automatically as well, but since they're TS constructs and not ES6, I think it's okay to force more discipline in their usage.

Something I've been considering is also rewriting namespaces as interfaces in the generated .d.ts in this way to further flatten/unify the types, but this... might? not strictly be needed. I haven't come up with a strong case for it.

@danquirk danquirk added Suggestion In Discussion labels Aug 25, 2015
@weswigham
Copy link
Author

@weswigham weswigham commented Aug 27, 2015

I realize that I've forgotten to propose a way to define the entrypoint module (as is possible in #4434), and I suppose that warrants discussion, too.

The entrypoint is the location which (in the above example) TS considers the 'top level' for flattening types (in this example, a.ts). Ideally, this is the entrypoint to your library or application. All of the dependent files in a well-formed module-based TS project should usually be accessible from the entrypoint via either imports or triple-slash refs... however In TS's default mode of operation for the --module flag, TS ignores most relationships between included files and compiles them mostly separately, resulting in separate js and dts files for each. Like the proposal in #4434, dts bundling may also make sense to require a --bundleDefinitions [entrypoint] flag, like how bundling module sources could require the --bundle [entrypoint] flag.

On the other hand, rather than add a new compiler flag, we could consider all files specified as 'root' files as entrypoints when compiling with --module and output a definition file for each of them. (Meaning that all other files need to be accessed by those root files and compiled via that relationship.) Conceptually, we could do the same thing with #4434, rather than having the --bundle argument to specify an entrypoint. This does lose the meaning of the --outFile argument, however, since there are suddenly multiple output files (one for each root file) and none of them correlate to the --outFile parameter... So maybe it's best to not try to rely on traversing the dependencies ourselves and require an extra parameter specifying the entrypoint.

@NoelAbrahams
Copy link

@NoelAbrahams NoelAbrahams commented Aug 27, 2015

@weswigham,

I'm generally in favour of this proposal. I think this feature would help to formalise the idea of a project in Visual Studio (i.e. types defined in namespaces compiled into a a single output file, that can then be referenced by other projects).

One of the problems with a single output file is that it makes life difficult when debugging in the browser. See #192 (comment). Ideally something can be worked out for that as well.

@weswigham
Copy link
Author

@weswigham weswigham commented Aug 27, 2015

#3159 implements most of this proposal, though it restricts it to ES6 or commonjs modules, uses slightly different terminology for tsc command line flags. and omits the type flattening aspect of this proposal. At the very least, it's probably an excellent starting point for talking about this.

@alexeagle
Copy link

@alexeagle alexeagle commented Sep 12, 2015

some notes from doing this for Angular:

  • should also report a semantic error when a symbol is re-exported to an entry point, but a dependent symbol is not (for example, declared supertypes, parameter types)
  • we like to emit a namespace as well as a module, allowing users to use symbols from that global namespace without any import statements (eg. useful for ES5 users in VSCode to get intellisense).
  • we might want to be able to use this typings bundle feature without having to use the runtime emit bundling feature from #4434 - for example if there are some errors producing a working emit because of private APIs, it would be nice if we could still use this to produce the public API doc.
  • We like to preserve some comments so that tools can show inline doc when showing eg. a completion tooltip. Should probably explicitly mention comment handling in the proposal.

@alexeagle
Copy link

@alexeagle alexeagle commented Sep 14, 2015

We can't drop our current .d.ts bundler without constructor visibility: #2341

@ffMathy
Copy link

@ffMathy ffMathy commented Dec 31, 2015

What is the current state of this issue?

@mhegazy
Copy link

@mhegazy mhegazy commented Jan 6, 2016

It is still in discussion. we have a PR for it, but there are still some open questions.

@ffMathy
Copy link

@ffMathy ffMathy commented Jan 6, 2016

Can you point to this pull request?

@mhegazy
Copy link

@mhegazy mhegazy commented Jan 6, 2016

I should clarify, the concatenation is already done in: #5090
The remaining piece is the flatting, here is the PR: #5332

@PavelPZ
Copy link

@PavelPZ PavelPZ commented Mar 11, 2016

I think that d.ts bundle works very nice, see angular#5796.

Thanks a lot for it.

@heruan
Copy link

@heruan heruan commented Apr 7, 2016

I jumped to/from many issues regarding this "theme". Is this the right one where discuss?

In my opinion the compiler should be able to produce also separate bundles, like this:

{
    "...": "...",
    "moduleDeclarationOutput": "./modules",
    "modules": {
        "module-a": {
            "from": "./src/module-a/index.ts"
        },
        "module-b": {
            "from": "./src/module-b/index.ts"
        }
    }
}

having then in ./modules/module-a.d.ts the definitions of types exported by ./src/module-a/index.ts and respectively for "module-b".

@davismj
Copy link

@davismj davismj commented May 16, 2016

requesting an update on this please

@mhegazy
Copy link

@mhegazy mhegazy commented May 16, 2016

nothing done since last update. no plans to do anything for TS 2.0 at this point.

@davismj
Copy link

@davismj davismj commented May 16, 2016

@weswigham What do you think would be a best strategy in light of this? Would you recommend building a custom TypeScript with your PR?

@weswigham
Copy link
Author

@weswigham weswigham commented May 17, 2016

I wouldn't recommend it, no. TBH, with recent changes to allow module
augmentation and even proposals for relative paths in module augmentation,
we may have a better way to do this now which can even preserve secondary
entry points.

On Mon, May 16, 2016, 6:24 PM Matthew James Davis notifications@github.com
wrote:

@weswigham https://github.com/weswigham What do you think would be a
best strategy in light of this? Would you recommend building a custom
TypeScript with your PR?


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#4433 (comment)

@davismj
Copy link

@davismj davismj commented May 18, 2016

Any other options, then?

@andyfleming
Copy link

@andyfleming andyfleming commented Oct 22, 2019

Also still looking for support for this.

In addition to outFile, it might be nice to specify outModule if you want the name to be different.

{
  "compilerOptions": {
    "module": "commonjs",
    "declarations": true,
    "out": "./types/lib.js",
    "outModule": "@my-org/example-package"
  }
}

One other alternative to consider would be to declare the module based on the name field in package.json.

@millsp
Copy link

@millsp millsp commented Nov 2, 2019

I wrote a small script to generate a bundle of your project's declarations:

#!/bin/bash

mkdir -p out

npx tsc ./src/index.ts --declaration --emitDeclarationOnly --out out/index.js --module amd &&

echo "
declare module 'YOUR_PACKAGE_NAME' {
    import main = require('index');
    export = main;
}
" >> out/index.d.ts

It outputs to out, change it as needed, and replace YOUR_PACKAGE_NAME.

What this script does NOT do is to bundle other projects declarations.

@jeremyben
Copy link

@jeremyben jeremyben commented Nov 27, 2019

Here's my take on it : bundle option with tsc-prog.

Faster than others because it bundles in memory, by intercepting emitted ".d.ts" files during the initial build.

Nevertheless, this is a complicated issue. Corner cases everywhere: name conflict (with global symbols and local ones), namespaces import and re-export, external libraries, json files, import type, etc. Just to name a few I tried to handle properly.

@gpickell
Copy link

@gpickell gpickell commented Mar 18, 2020

If the intent is to offer a single "d.ts" for a single-script library, then that is a complicated problem. As a thought, a lot of d.ts files are becoming quite large. Maybe it is not the end of the world to keep the d.ts files laid out as per the default. However, maybe instead introduce a "friend" concept for d.ts files.

Idea shopping...

lib/src/internal.ts:
/// <friend modules="~" />; // Says that the only src relative modules can reference this one.


index.ts:
import ... from "./internal"; // Works because ./ matches ~ (src root)
import ... from "../folder/internal"; // Works because ../ matches ~ (src root)

(so far the ts would only perform syntax validation)

app.ts:
import ... from "lib"; // References index.d.ts so good.
import ... from "lib/internal"; // Error: type checker sees that this module is not a friend.

Advantages:

  • Promotes size sanity of d.ts files.
  • Keeps d.ts files laid out as per the src.
  • Completely compatible with current declaration parsing.
  • The friend concept can extend across NPM projects potentially enriching options there.
  • There is no need to add any new configuration options.
  • Multiple "entry points" are valid with this approach without any real additional concepts.

Cons:
Maybe not what people really want.

@kripod
Copy link

@kripod kripod commented May 26, 2020

Hello,

Thanks to everyone for the valuable conversations above. I would like to ask whether any method is known for bundling declaration maps, especially when Microsoft’s API Extractor is in use.

@octogonz
Copy link

@octogonz octogonz commented May 26, 2020

I would like to ask whether any method is known for bundling declaration maps, especially when Microsoft’s API Extractor is in use.

@kripod This forum is for the TypeScript compiler. To ask an API Extractor specific question, please create an issue for its GitHub project: https://github.com/microsoft/rushstack/issues

You will need to provide more detail explaining how the declaration maps would be used. Thanks!

@timsuchanek
Copy link

@timsuchanek timsuchanek commented Sep 10, 2020

For anyone wondering, which of the mentioned solutions works.
I tried out a few and this one https://github.com/Swatinem/rollup-plugin-dts worked the easiest for me.
Just give it 5 min.
Even if you're not a rollup user, for this single feature it's worth it.
Just use the default config that is provided:

import dts from "rollup-plugin-dts";

const config = [
  // …
  {
    input: "./my-input/index.d.ts",
    output: [{ file: "dist/my-library.d.ts", format: "es" }],
    plugins: [dts()],
  },
];

export default config;

Install the rollup cli:

npm i -g rollup

and execute

rollup -c

We're even using it with the respectExternal option to pull in external types as we're bundling other libraries as well.

@germanz
Copy link

@germanz germanz commented Jan 5, 2022

For the webpack users, there is also this option via plugin npm-dts-webpack-plugin:

const NpmDtsPlugin = require('npm-dts-webpack-plugin')

module.exports = {
  ......
  plugins: [
    new NpmDtsPlugin({
      logLevel: 'debug'
    })
  ],
  ......
}

macarc pushed a commit to macarc/marender that referenced this issue Jan 15, 2022
Typescript doesn't seem to be able to output type declarations the way I
want (-> microsoft/TypeScript#4433), so we'll
just have to manually do it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
In Discussion Suggestion
Projects
None yet
Development

No branches or pull requests