@andrewbranch Here are the steps I've been using to test local ts changes on a local web build of VS Code:

1. Build a copy of TS locally

2. In vscode, comment out the CopyPlugin section of this build file (this is the rule that runs the minifier):

   https://github.com/microsoft/vscode/blob/cde5781978134c0091d28a4f11b45b2f08412b4f/extensions/typescript-language-features/extension-browser.webpack.config.js#L63

3. Build VS Code for web: yarn watch-web

4. Link your built TS Server in place of the VScode one:

   # In the VSCode repo
   cd extensions/typescript-language-features/dist/browser/typescript
   ln -s ~/projects/typescript/built/local/tsserver.js tsserver.web.js
   

5. Now run for web: ./scripts/code-web.sh. This should open http://localhost:8080

I'm not sure I understand why a VFS would need an identifier - do we anticipate having multiple or mixing VFS and actual FS access? Why wouldn't this command just say "this is your view of the FS until further notice"?

Possibly relevant: I believe the tests already virtualize FS access.

Do we have a sense of how much slower this would be than a more specialized API saying "this tarball if your FS"? If it's substantial, we might want a "payload kind" property.

I'm not sure I understand why a VFS would need an identifier - do we anticipate having multiple or mixing VFS and actual FS access? Why wouldn't this command just say "this is your view of the FS until further notice"?

On the basic web, I think everything will be on the VFS. But on desktop VS Code, you can end up in situations where some files exist on disk and some exist on a virtual file system. This could happen if you create a workspace that has one folder from disk and one from somewhere like GitHub or one drive for example

Do we have a sense of how much slower this would be than a more specialized API saying "this tarball if your FS"? If it's substantial, we might want a "payload kind" property.

My suspicion is that in most workspaces, the number of js/ts files we need to send over will be pretty small (< 100). For something like the typescript project, we would have to send a lot of files though. I don't expect the data transfer to be the main bottleneck but we will need to test this (we could also try to optimize it using transferables if needed)

We can also have a cap of the number of files we send if we do run into issues

I don't expect the data transfer to be the main bottleneck but we will need to test this (we could also try to optimize it using transferables if needed)

A qualified guestimate is "F1 > Measure Extension Host Latency". Take it with a grain of salt but on vscode.dev with Safari I see up/download speeds ~2000Mbs.

I might also be noteworthy that a virtual file system is the more generic solution, e.g. vscode.dev also supports ADO and there no tarball is available. A VFS can abstract that away and give room for other optimisations like a browser-side git clone etc

I'm going to read through the proposal and just plop a bunch of discussion points that I want to work through. Please note it's all constructive, I'm just trying to think through the scenarios!

The above proposal takes a flat list of files similar to update opened. If we think it would be more convenient, we could instead take a tree-like structure.

One of the concerns here is deciding what is explicit and implicit. We should get a sense of when it is and isn't useful to make these distinctions. For example, you create a file called /foo/bar/baz.ts. That presumably means that there's a root directory at /, it contains a folder called foo, that contains a folder called bar, and that contains a file called baz.ts

We can "do the right thing", but off the bat this can't support things like symlinks, other file-like abstractions, weird niche systems that could allow files and folders with the same name, etc. The more we see vscode.dev doing in the future, the more this sort of stuff will matter.

I think for me, the biggest concern is symlinks. If you're planning on simulating npm install-like scenarios, this will be good to have.

{ path: "/workspace/index.js", contents: "import * as abc from './sub/abc'" },

On this note, if we do go beyond directory and path, each entry having its own tag field instead of a bunch of optional properties like contents would be nice too. This might make it easier to guide the API into the pit of success.

In other words, what I'd hope for with this is that it's a discriminated union type. Maybe I'm going overkill.

updateFileSystem {
    fileSystem: 'memfs',
    opened: [
        { path: "/workspace/index.js", contents: "import * as abc from './sub/abc'" },
        { path: "/workspace/src/abc.js", contents: "export const abc = 123;" },
        { path: "/workspace/test/xyz.test.ts", contents: "..." },
    ]
}

I know this might just be a rough sketch now, but should this be created and not opened?

updateFileSystem {
    fileSystem: 'memfs',

I dunno if @amcasey already asked this, but what else goes in place of memfs?

On the basic web, I think everything will be on the VFS. But on desktop VS Code, you can end up in situations where some files exist on disk and some exist on a virtual file system. This could happen if you create a workspace that has one folder from disk and one from somewhere like GitHub or one drive for example

This leads me into a few questions:

• Who's the source of truth for TSServer? Does memfs always take precedence?
• If we have the file system available, do we ever watch for a file of the same name on disk?
• Relatedly, can projects contain files from both the VFS and disk? Or are they disjoint?

A qualified guestimate is "F1 > Measure Extension Host Latency". Take it with a grain of salt but on vscode.dev with Safari I see up/download speeds ~2000Mbs.

That's cool! Still, we'd need to be a little careful I assume. Is a worker able to hold all this in memory? We'll need to make sure no client ever says "we just sent you 2 GB, good luck, have fun".

Let me ask a weirder question: why wouldn't VS Code on desktop always talk to TypeScript through a VFS? We already have twice the number of file-watchers that we need.

I think the answer is that language servers contain the necessary logic to filter down files - but if transfer speed is not a problem, can you just toss everything over to TS all the time, regardless of vscode.dev vs. VS Code desktop?

Feels like a crazy suggestion, but maybe it's an assumption worth revisiting!

I think the answer is that language servers contain the necessary logic to filter down files - but if transfer speed is not a problem, can you just toss everything over to TS all the time, regardless of vscode.dev vs. VS Code desktop?

Uh, and I guess editors also don't want to traverse and read every single file in a workspace. 🙄🤦‍♂️

But then if that's a concern, how would we get ATA to be speedy?

I had similar questions about whether this can/should be enabled alongside a real FS on desktop. That could open up a lot of possibilities for VS Code extensions / LS plugins, but could also become a source of hard-to-diagnose bugs, and could make it difficult for us to update the API if it saw a surge of third-party adoption.

memfs:/workspace/index.js

Here's another thing - is this a new URI protocol style we would want to respond to? Would it make more sense for the request to talk about files being in memfs: in the first place?

updateFileSystem {
    created: [
        { path: "memfs:/workspace/index.js", contents: "import * as abc from './sub/abc'" },
        { path: "memfs:/workspace/src/abc.js", contents: "export const abc = 123;" },
        { path: "memfs:/workspace/test/xyz.test.ts", contents: "..." },
    ],
    deleted: [],
    updated: [
        { path: "memfs:/workspace/test/xyz.test.ts", contents: "..." }
    ]
}

Thanks for the feedback. Just a few thoughts on some of these points:

One of the concerns here is deciding what is explicit and implicit.

I think a more explicit, tree-like structure for the file system has a few benefits but may be more difficult to implement/work with. For example, using a tree would let us express empty directories but then we'd also have to tell TS when a directory is deleted. Let's talk about this more

We can "do the right thing", but off the bat this can't support things like symlinks, other file-like abstractions, weird niche systems that could allow files and folders with the same name, etc. The more we see vscode.dev doing in the future, the more this sort of stuff will matter.

In general, I think we should try aligning with VS Code's model of file systems. That way we don't have to worry about aspects such as case-sensitive vs case-insensitive file names

Symlinks are interesting though. I believe (but haven't actually tested this yet) that VS Code's virtual file system supports them. Something to consider for the design even if they are not supported in a v1

This leads me into a few questions:

1. Who's the source of truth for TSServer? Does memfs always take precedence?
2. If we have the file system available, do we ever watch for a file of the same name on disk?
3. Relatedly, can projects contain files from both the VFS and disk? Or are they disjoint?

1. IMO for TS Server, the source of truth is: first the opened file and then the VFS.

   TSServer should never try updating its model of the VFS based on changes to opened files

2. No, I don't think this should happen.

   With my thinking, we should never end up in a case where a file exists on both disk and in the VFS. If a client does wish to surface an on-disk file through the VFS, it must always talk to TSServer using the VFS

3. Yes I think we should allow this, but we'd have to figure out what that actually means for TS projects.

   A use case for VS Code would be: I'm browsing a normal on-disk repo on desktop VS Code and then try opening a JS file that's contained inside of an archive, such as a zip. This file could be surfaced to TSServer using a VFS

Let me ask a weirder question: why wouldn't VS Code on desktop always talk to TypeScript through a VFS? We already have twice the number of file-watchers that we need.

Interesting thought!

With this proposal, the downside is that we have to eagerly sync over the VFS contents. In previous discussions however, we did talk about having TS be able to delegate file system operations back to VS Code. The main problem we ran into is that TS expects file reads to be sync, and there's not a good way to implement that since we'd have to communicate back and forth with the VS Code process

Here's another thing - is this a new URI protocol style we would want to respond to? Would it make more sense for the request to talk about files being in memfs: in the first place?

Yes I like that, although this protocol may need to completely change if we decide we need a more explicit api

A use case for VS Code would be: I'm browsing a normal on-disk repo on desktop VS Code and then try opening a JS file that's contained inside of an archive, such as a zip.

I want to dig into that more to better clarify how to model mixed virtual/non-virtual files in the same project.

Let's say you have a project on disk with /proj/foo.ts and /proj/bar.ts. A client editor says "Hey, /proj/bar.ts exists in a virtual filesystem."

If you allow these files to mix, then my assumption is that the virtual file system has to take precedence for all LS operations. But I don't know what TSServer would do with the old SourceFile of /proj/bar.ts. I also don't know if there's a case where some projects see the on-disk /proj/bar.ts and others see the in-memory /proj/bar.ts - I think

To continue, let's say the client editor says "/proj/bar.ts no longer exists in the virtual filesystem." I would assume that TypeScript still has to re-resolve /proj/bar.ts on disk in some way for at least one of the projects that originally contained it, right?

These might be weird edge-cases, but I want us to have a good mental model and some well-understood behavior here.

The main problem we ran into is that TS expects file reads to be sync, and there's not a good way to implement that since we'd have to communicate back and forth with the VS Code process

Yeah, that is weird - but this current idea might be less chatty than something like that. This isn't something we have to do for v1 of course.

One thing that came up in our sync today was whether we had to proactively defend against extremely large files, or too many files that are irrelevant to TS. Our thinking is that we should stick to something simple at first, but that we have room to grow and improve the experience here. One idea was to have editors send over truncated files, or omit their contents entirely, and mark those files as "proactively omitted". The server could then signal "hey, we actually did need these files if they're still around", and an editor could choose to provide them.