RFC: allowing standalone `.d.ts` emit through explicit type annotations (--isolatedDeclarations, --noTypeInferenceOnExports?) #47947

mprobst · 2022-02-18T10:26:23Z

Suggestion

TypeScript supports relying on type inference to produce (parts of) the API of a module. E.g. users can write (sorry, slightly contrived):

// in counter.ts:
import {Splitter} from 'textutils/splitter';
export function countParts(x: string) {
  return new Splitter(x).splitWords().size();
}

Note how you need to specify the type of x (otherwise it degenerates to any), but can leave out the return type of lengthOf if you like. TypeScript will infer the type, potentially using type information from the file's (transitive) dependencies.

This causes two problems.

Readability. It is difficult to understand what the return type of countParts will be. This is purely a stylistic issue that could be fixed with a lint check (though there is some complexity e.g. due to typeof).

Compilation performance/parallelism.

Imagine you're using project references, and you have a dependency structure:

app <-- counter <-- textutils/splitter

To compile, we need to first compile textutils/splitter, wait for that to complete, e.g. 5s, then compile counter, wait e.g. 3s, then compile app (6s). Total compilation wall time is |app| + |counter| + |textutils/splitter|, in our example 5s + 3s + 6s = 16 seconds.

Now assume we could produce .d.ts files without requiring transitive inputs. That'd mean we could, in parallel, produce the .d.ts files for textutils/splitter, counter (and app, though we don't need that). After that, we could, in parallel, type check and compile textutils/splitter, counter, and app. Assuming sufficient available parallelism (which seems reasonable, given how common multicore CPUs are), total compilation wall time is the maximum time to extract .d.ts files, plus the time for the slowest compile. Assuming .d.ts extraction is purely syntactical, i.e. does not need type checking nor symbol resolution, it shouldn't add more overhead than a few hundred ms. Under these assumptions, the wall time to wait for the project to compile would be 500 ms + 6s = 6.5 seconds, i.e. more than a x2 speedup.

The problem with that is that we cannot produce .d.ts files without running full type checking, I believe purely due to type inference.

RFC thus: I wonder if this would sufficiently motivate the ability to restrict using type inference in exported API?

E.g. we could have a noTypeInferenceOnExports compiler flag, that would allow TypeScript to parallelise emitting .d.ts across project references, and then parallelize type checking.

The counter point is that projects that experience slow builds in edit refresh situations might instead want to turn off type checking entirely for their emit, at least on critical paths. However that means users do not see compilation results, and produces additional complexity (e.g. when and how to report type checking failures).

Impact

We've run some statistics internally at Google on build operations.

As one would expect, this change has little impact on most incremental "hot inner loop" builds, as those typically just re-type check a single file and produce no .d.ts change at all, so caching saves us from long build chains. We're seeing ~20% wall time improvements in the 90th percentile across all builds involving TypeScript.

However the impact on slower builds is more substantial. For a sample "large" project that sees both slow individual compiles and a long dependency chain, we see ~50% improvement in the 90th percentile, and 75% in the 99th percentile (which is representative of CI style "cold" builds with little caching).

🔍 Search Terms

performance compilation parallelism inference declarations

✅ Viability Checklist

My suggestion meets these guidelines:

This wouldn't be a breaking change in existing TypeScript/JavaScript code
This wouldn't change the runtime behavior of existing JavaScript code
This could be implemented without emitting different JS based on the types of the expressions
This isn't a runtime feature (e.g. library functionality, non-ECMAScript syntax with JavaScript output, new syntax sugar for JS, etc.)
This feature would agree with the rest of TypeScript's Design Goals.

The text was updated successfully, but these errors were encountered:

fatcerberus · 2022-02-19T02:37:03Z

Even if you could force explicit type annotations on everything, I don’t think that would be enough to enable parallel compilation, since the structural typing means the compiler has to know what the imported types contain in order to check them.

mprobst · 2022-02-19T09:31:07Z

Bruce: the assumption would be that you generate `.d.ts` files in one step, which is purely syntactical, i.e. no type checking happens here. And then you type check all sources (`.ts` and `.d.ts`) in a second step. In the second step, you have all `.d.ts` files that you generated in the first step available - so you can perform all checks that TS does as it would today.

robpalme · 2022-02-22T22:36:08Z

Thanks for raising this issue. This is a feature/area I had been considering prototyping for a while. It sounds like we're thinking of the same thing but I'll elaborate here so you can verify.

Feature Description

The key goal is to allow a declaration *.d.ts to be generated from a single *.ts file, without the need to examine the dependency tree. This implies that any types needed from dependencies will be imported by the generated *.d.ts. That's a major change from today's declaration generation that sometimes inlines (duplicates) those resolved types into the generated declaration rather than referencing them via imports.

This kind of standalone declaration generation isn't possible for the full set of TypeScript source files today. An example problematic case is when an exported type is computed based on types originated in dependencies. Declaration files can't express transitive concepts such as typeof a + typeof b so the emitted declaration is always a resolved type today.

import { a, b } from "./dependency";
export const sum = a + b;    // declaration emit will resolve this to a singular string or number

Potentially new declaration syntax could be added to mitigate this. But a general solution is to constrain the set of TypeScript that can be authored. Think of it as introducing stronger linting rules. When this need arose previously to permit standalone per-file TS->JS compilation, the isolatedModules option was introduced to constrain the source, forcing the user to write more explicit code in some cases. So I think the natural option name for this new constrained mode applied to declaration generation would be isolatedDeclarations

Pros & Cons

As you say, this feature will permit parallelisation of declaration emit. It will also reduce the blast radius of declaration regeneration needed when editing a single file. Another less obvious benefit is that it will improve type-checking performance for consumers of otherwise bloated declaration files - by eliminating the (increasingly rare) edge cases where excessive inlining super-sizes the declaration files. And when declaration emit is decoupled from type-checking, it opens the door for high-performance declaration emit tools in other languages, e.g. Rust/Go/Zig.

A downside of forcing isolation of declaration files is that it may increase the count of file accesses for anyone consuming that declaration file set during type-checking. Per-file overheads are noticeable particularly on Windows and particularly when malware scanners intercept the file access. This can be mitigated by declaration bundling.

mprobst · 2022-02-23T12:33:22Z

@robpalme yes, what you're writing here matches my understanding. And indeed, isolatedDeclarations is a good name.

One thing I don't follow on: why do you think this will increase the number of file accesses needed during type checking? Shouldn't the compiler just read the exact same set of .d.ts files as before?

robpalme · 2022-02-23T13:34:14Z

One thing I don't follow on: why do you think this will increase the number of file accesses needed during type checking? Shouldn't the compiler just read the exact same set of .d.ts files as before?

My fault for not being clear. This proposal is really two things:

Stricter linting errors
- Achieved by either --isolatedDeclarations in tsc, or perhaps externally via a standalone ESlint rule
A new per-file declaration generator
- Achieved either in tsc, or perhaps externally via a new standalone tool

It is possible to deliver (1) without (2). But delivering (2) requires (1).

(2) may entail more file accesses during checking, as it guarantees no inlining of types. Last I checked, the checker is lazy and only loads imports that are truly needed to check something. Inlining, as opposed to referencing, will lead to more cases where that laziness pays off and means a dependency file does not need to be loaded. This is a minor/rare case that shouldn't really sway anything.

RyanCavanaugh · 2022-02-23T20:57:17Z

I would frame it this way:

isolatedModules means a syntax-only tool can be guaranteed to do single-file transpilation correctly
isolatedDeclarations means a syntax-only tool can be guaranteed to do single-file .d.ts emit correctly

The potential perf gains here are indeed extremely large if we think about non-error-checking scenarios.

We'll have to consider what the edge cases are where the syntactic rules might be sufficient to capture this invariant, but it's a very interesting proposal.

Conaclos · 2022-03-13T11:42:12Z

This looks like type-first mode of Flow. They obtained a great perf boost.

dragomirtitian · 2022-03-14T17:19:19Z

I started doing some experimentation with isolatedDeclarations and here are my findings so far

mprobst changed the title ~~RFC: allowing standalone .d.ts emit through explicit type annotations (--noTypeInferenceOnExports?)~~ RFC: allowing standalone .d.ts emit through explicit type annotations (--isolatedDeclarations, --noTypeInferenceOnExports?) Feb 23, 2022

RyanCavanaugh added In Discussion Suggestion labels Feb 23, 2022

microsoft / TypeScript Public

RFC: allowing standalone `.d.ts` emit through explicit type annotations (--isolatedDeclarations, --noTypeInferenceOnExports?) #47947

RFC: allowing standalone `.d.ts` emit through explicit type annotations (--isolatedDeclarations, --noTypeInferenceOnExports?) #47947

mprobst commented Feb 18, 2022 •

edited

fatcerberus commented Feb 19, 2022

mprobst commented Feb 19, 2022

robpalme commented Feb 22, 2022

mprobst commented Feb 23, 2022

robpalme commented Feb 23, 2022

RyanCavanaugh commented Feb 23, 2022

Conaclos commented Mar 13, 2022 •

edited

dragomirtitian commented Mar 14, 2022

microsoft / TypeScript Public

RFC: allowing standalone .d.ts emit through explicit type annotations (--isolatedDeclarations, --noTypeInferenceOnExports?) #47947

RFC: allowing standalone .d.ts emit through explicit type annotations (--isolatedDeclarations, --noTypeInferenceOnExports?) #47947

Comments

mprobst commented Feb 18, 2022 • edited

Suggestion

Impact

🔍 Search Terms

✅ Viability Checklist

fatcerberus commented Feb 19, 2022

mprobst commented Feb 19, 2022

robpalme commented Feb 22, 2022

Feature Description

Pros & Cons

mprobst commented Feb 23, 2022

robpalme commented Feb 23, 2022

RyanCavanaugh commented Feb 23, 2022

Conaclos commented Mar 13, 2022 • edited

dragomirtitian commented Mar 14, 2022

RFC: allowing standalone `.d.ts` emit through explicit type annotations (--isolatedDeclarations, --noTypeInferenceOnExports?) #47947

RFC: allowing standalone `.d.ts` emit through explicit type annotations (--isolatedDeclarations, --noTypeInferenceOnExports?) #47947

mprobst commented Feb 18, 2022 •

edited

Conaclos commented Mar 13, 2022 •

edited