Mcp apps by MatthewKhouzam · Pull Request #17489 · eclipse-theia/theia

MatthewKhouzam · 2026-05-12T03:19:48Z

What it does

This patch adds support for MCP (Model Context Protocol) apps. It has only been tested with Ollama local.

How to test

Open an MCP app (e.g. TMLL)
Query an app response
A video will be placed in the comments

Follow-ups

Breaking changes

This PR introduces breaking changes and requires careful review. If yes, the breaking changes section in the changelog has been updated.

Attribution

Review checklist

As an author, I have thoroughly tested my changes and carefully followed the review guidelines
User-facing text is internationalized using the nls service (for details, please see the Internationalization/Localization section in the Coding Guidelines)

Reminder for reviewers

As a reviewer, I agree to behave in accordance with the review guidelines

mcp-app.mp4

MatthewKhouzam · 2026-05-12T03:23:38Z

I cannot say this is THOUROUGHLY reviewed, as I lack most llm access.

@ndoschek I am leaving this in your hands. Sorry. (Rejecting the patch is 100% acceptable :) )

MatthewKhouzam · 2026-05-12T13:54:15Z

eclipse-tmll/tmll#17 this patch can be used for testing.

Add support for rendering interactive HTML content returned by MCP tools in the chat response view via a sandboxed iframe. - Add ToolCallHtmlAppResult type and isToolCallHtmlAppResult type guard - Handle 'html' content type in MCP frontend service tool handler - Add McpAppFrame component with sandboxed iframe (allow-scripts only) - Render html tool results in ToolCallPartRenderer - Add unit tests for type guard, content mapping, and srcDoc injection This code was written with the assistance of claude opus 4.6 Signed-off-by: Matthew Khouzam <matthew.khouzam@ericsson.com>

- Handle 'html' content type in Anthropic model (falls back to text) - Parse JSON-wrapped HTML results from MCP tool call text responses - Add test for iframe-ready HTML app result validation This change was made with the assistance of claude opus 4.6 Signed-off-by: Matthew Khouzam <matthew.khouzam@ericsson.com>

Resolve anyOf types in tool parameter conversion by picking the first non-null type, instead of silently skipping unsupported parameters. This code was made with the assistance of claude sonnet 4.6 Signed-off-by: Matthew Khouzam <matthew.khouzam@ericsson.com>

- Update copyRipgrep to resolve from @vscode/ripgrep-{platform}-{arch} - Add packageManager field to package.json The commit message was normalized with claude opus 4.6 Signed-off-by: Matthew Khouzam <matthew.khouzam@ericsson.com>

…gle models Extract text/html content from ToolCallContent results instead of blindly JSON.stringifying, consistent with Anthropic's approach. This code was made with the assistance of claude opus 4.6 The author does not have access to the AI platforms to test this patch Signed-off-by: Matthew Khouzam <matthew.khouzam@ericsson.com>

ndoschek · 2026-07-13T16:21:56Z

Hi @MatthewKhouzam, thanks a lot for this, really nice addition that levels up the AI chat experience! 🎉
I rebased onto the latest master to resolve the conflicts and tested it with the TMLL trace-server MCP, and it works nicely with Anthropic and that one works fine so far (still need to test the other ones).

I do have a little feedback, but only smaller UI/UX stuff, nothing major. How would you like to proceed, should I just take care of it directly or leave it as review comments? Fine either way for me, we can also quickly discuss it in the dev call tomorrow for example!

MatthewKhouzam · 2026-07-13T18:33:26Z

Let's discuss tomorrow. But I have a bit of breathing time now and would like to contribute to theia :)

ndoschek

Thanks again for tackling this! 🎉
I tested it with the TMLL trace-server MCP and the main providers (Anthropic, OpenAI, Google, Copilot, Custom OpenAI) and the core flow works well!
I added a few inline comments, but nothing major.

I also have a few general comments:

App HTML sent back to the model (Response API path): the full app HTML gets serialized back into the model context and blows the context window. It also hits the Response API path in openai-response-api-utils.ts (the function_call_output serialization), which the custom OpenAI endpoint uses, where I got errors like: 400 This endpoint's maximum context length is 262144 tokens. However, you requested about 290809 tokens.... Fix is probably the same as the inline comments: send the model a compact placeholder for html results (keep the full HTML only for the iframe). Flagging this here since that file isn't touched by this PR.
OpenAI strict schema: With a real MCP server, official OpenAI 400s the whole request because strict mode can't represent open-ended-map params (e.g. TMLL's series: dict becomes additionalProperties) and all tools are validated up front, so one bad schema takes everything down. This lives in openai-response-api-utils.ts (not touched by this PR) and should be resolved by PR #17754, which removes the strict conversion entirely. So I will quickly re-test with that once it lands rather than fixing it here.
edit: I just tested it with the linked PR and this fixes this problem already 👍 🎉

A few things I'd noticed but are fine IMO to treat as follow-ups:

Auto-expand app results so they're not hidden in the collapsed tool call
Generalize the sandboxed iframe into a reusable component (mermaid could then adopt it and drop DOMPurify)
Frame toolbar + light/dark theming injected into the app
Decide on allow-downloads (Plotly's export button is currently dead under the sandbox)
Upstream: TMLL should also return a text/data summary alongside the app so the model can reason about the result

ndoschek · 2026-07-14T15:07:58Z

This was probably just an oversight, since we are npm only the commit introducing this change should be dropped.

ndoschek · 2026-07-14T15:25:33Z

+    if (isToolCallContent(result)) {
+        return result.content.map(c => {
+            if (c.type === 'text') { return c.text; }
+            if (c.type === 'html') { return c.html; }


This returns the full app HTML straight back to the model. MCP apps inline whole libraries (TMLL bundles Plotly, hundreds of KB), and the tool result is re-sent on every follow-up turn, so it blows the context window fast: I hit 400 maximum context length (~290k tokens) on OpenAI and watched Gemini loop and re-emit the entire Plotly bundle. The HTML is only needed for the iframe; for the model, please send a compact placeholder instead (e.g. [interactive app displayed to the user: <title>]) and still pass through any text/error parts so a server-provided summary survives. The same issue exists in every provider's tool-result serialization (linked below).

ndoschek · 2026-07-14T15:26:14Z

+    if (isToolCallContent(content)) {
+        const text = content.content.map(c => {
+            if (c.type === 'text') { return c.text; }
+            if (c.type === 'html') { return c.html; }


same as the OpenAI comment: this returns the full app HTML to the model. Replace with the compact placeholder

ndoschek · 2026-07-14T15:27:09Z

+    if (isToolCallContent(result)) {
+        return result.content.map(c => {
+            if (c.type === 'text') { return c.text; }
+            if (c.type === 'html') { return c.html; }


same as the OpenAI comment: this returns the full app HTML to the model. Replace with the compact placeholder

ndoschek · 2026-07-14T15:29:23Z

+    return (
+        <div className='theia-mcp-app-container'>
+            {title && <div className='theia-mcp-app-title'>{title}</div>}
+            <iframe


Since the app runs with allow-scripts and no CSP, the sandboxed content can make arbitrary outbound network requests and load remote resources. Could we tighten this before merge by adding a CSP <meta> to the srcDoc so the frame can't reach the network or pull in remote code?
The mermaid renderer already restricts remote resource loading, so it'd be good to stay consistent. Something like default-src 'none'; script-src 'unsafe-inline'; style-src 'unsafe-inline'; img-src data: blob:; connect-src 'none'; ..., plus allow="" on the iframe. Apps that need runtime fetch could opt in explicitly.

ndoschek · 2026-07-14T15:34:18Z

+}).observe(document.documentElement);
+</script>`;
+
+    function buildSrcDoc(html: string): string {


This test defines its own buildSrcDoc copy instead of importing McpAppFrame, so it passes even if the real component breaks. Could it exercise the actual code?

ndoschek · 2026-07-14T15:34:44Z

-                                return { type: 'text', text: callContent.text };
+                            case 'text': {
+                                const text = callContent.text;
+                                if (text.startsWith('{')) {


Detecting an app via "text starts with { and parses to {type:'html'}" can misclassify legitimate JSON text output as an app iframe. Could we use a more explicit signal (e.g. a dedicated content type / MCP-UI resource) here?

ndoschek · 2026-07-14T15:35:24Z

+describe('MCPFrontendServiceImpl html content mapping', () => {
+
+    // Simulates the mapping logic from convertToToolRequest's default case
+    function mapContent(callContent: Record<string, unknown>): ToolCallContentResult {


Same here: mapContent is a hand-copied reimplementation of the conversion logic in mcp-frontend-service.ts, and it omits the new text to html branch, so the trickiest path is untested. Could this drive the real function instead?

ndoschek · 2026-07-14T15:36:15Z

 import { createProxyFetch } from '@theia/ai-core/lib/node';
 import { ollamaThinkParamFor } from './ollama-reasoning';

+function formatToolCallContent(result: ToolCallResult): string {


This flatten-to-string helper is duplicated across OpenAI/Copilot/Ollama (and inline in Google/Anthropic).
Would be worth extracting a single shared helper in ai-core so it stays consistent across providers.

ndoschek · 2026-07-14T15:36:57Z

+        : `${html}${RESIZE_SCRIPT}`;
+
+    return (
+        <div className='theia-mcp-app-container'>


theia-mcp-app-container isn't defined in any stylesheet, and the iframe uses inline style for width/border. Please move these to CSS classes (only the dynamic height needs to stay inline).

TheMatthew · 2026-07-15T02:33:00Z

only smaller UI/UX stuff, nothing major.

proceeds to find a bug that would cost thousands of euros. :)

github-project-automation Bot added this to PR Backlog May 12, 2026

github-project-automation Bot moved this to Waiting on reviewers in PR Backlog May 12, 2026

MatthewKhouzam force-pushed the mcp-apps branch from e1a0860 to 74338a1 Compare May 12, 2026 14:48

ndoschek assigned ndoschek and unassigned ndoschek May 13, 2026

ndoschek self-requested a review May 13, 2026 06:54

MatthewKhouzam added 5 commits July 13, 2026 16:49

ndoschek force-pushed the mcp-apps branch from 74338a1 to 86e8a99 Compare July 13, 2026 15:31

ndoschek requested changes Jul 14, 2026

View reviewed changes

github-project-automation Bot moved this from Waiting on reviewers to Waiting on author in PR Backlog Jul 14, 2026

ndoschek mentioned this pull request Jul 14, 2026

Remove strict schema conversion for OpenAI Response API #17754

Open

3 tasks

Uh oh!

Conversation

MatthewKhouzam commented May 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What it does

How to test

Follow-ups

Breaking changes

Attribution

Review checklist

Reminder for reviewers

Uh oh!

MatthewKhouzam commented May 12, 2026

Uh oh!

MatthewKhouzam commented May 12, 2026

Uh oh!

ndoschek commented Jul 13, 2026

Uh oh!

MatthewKhouzam commented Jul 13, 2026

Uh oh!

ndoschek left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TheMatthew commented Jul 15, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

MatthewKhouzam commented May 12, 2026 •

edited

Loading

ndoschek left a comment •

edited

Loading