Cross-Host Compatibility Testing for MCP Apps: ChatGPT, Claude, and Beyond (October 2026)

Cross-host compatibility testing catches bugs that only appear in specific MCP App hosts.
Your MCP App passes every test. Green across the board. You publish it for ChatGPT, someone installs the same server as a Claude Connector, and a card that looked polished in one host clips in the other. The tool call works, but the action button does nothing. Dark mode passes in your browser and fails inside the host iframe.
That is the core cross-host problem. MCP Apps are designed to be portable, but the app does not run in a browser you control. It runs inside a host-controlled iframe, with host-owned chrome, safe areas, display modes, CSS variables, bridge timing, and optional platform APIs.
TL;DR: Start with the MCP tool contract and a readable text response for hosts without UI support. Run one focused Playwright matrix across ChatGPT and Claude for display modes, themes, data states, and capability fallbacks. Use standard MCP Apps bridge methods for shared behavior, feature-detect optional host APIs, and keep separate visual baselines per host.
What Changed Since the First Wave of ChatGPT Apps
The official MCP Apps extension defines a portable pattern: a tool declares a ui:// resource in _meta.ui.resourceUri, the host calls the tool and fetches that resource, and the UI renders in a sandboxed iframe. The UI receives tool input and results through the host bridge, then sends actions back through that bridge.
OpenAI’s UI guide recommends shared MCP Apps fields and bridge methods for new UI. ChatGPT also exposes optional window.openai extensions for file handling, modals, and eligible checkout flows. OpenAI recommends detecting each extension and providing a fallback where practical, rather than branching on the product name.
Anthropic’s Claude connector guide covers remote MCP servers and optional MCP Apps UI. The same server may be called by a host that can render the UI and one that only consumes the tool response. Test both paths when you claim broad host compatibility.
That gives you two contracts to verify: the server response must stand on its own, and the interactive view must work when a host supports it.
Check the Protocol Before the Pixels
MCP Apps is an extension that hosts advertise during capability negotiation. A server can register UI-enabled tools for supporting hosts and still provide ordinary tools to hosts without UI support. In the UI path, content remains readable to the model, while structuredContent can provide data for the view. If a screenshot looks right but the text response is an empty placeholder, a text-only host has little to work with.
Before opening a browser, inspect one representative tool response and answer these questions:
- Does the tool have a stable input and output schema, and does
_meta.ui.resourceUripoint to a resource the host can fetch? - Can someone understand the result from
contentalone when no iframe is rendered? - Does
structuredContentcontain the fields the view needs, including explicit empty and error states? - Are UI-only actions exposed as app-visible tools where appropriate, and can the host reject them without corrupting the view?
The MCP Apps overview explains the tool-to-resource link and the text fallback. The official testing guide describes a reference host you can use to inspect a server outside your target product.
What Differs Between Hosts
Before writing tests, list the surfaces where a host can change behavior. The tool result can be valid and the UI can still fail because one of these surfaces changed.
| Surface | What can differ | What to test |
|---|---|---|
| Resource iframe | Sandbox, origin, allowed network targets, iframe sizing | Resource loads, CSP works, external assets render |
| Tool data | Timing of input/result delivery, _meta, structuredContent, empty states | Loading, approval, success, error, cancelled, empty |
| Host bridge | callServerTool, messages, model context, optional platform APIs | Accepted path, rejected path, missing capability path |
| Theme | CSS variables, dark palette, font stack, contrast | Light, dark, computed styles, screenshot baselines |
| Display mode | Inline, fullscreen, picture-in-picture, unsupported modes | Layout, overflow, mode request fallback |
| Safe area | Header, sidebar, mobile chrome, keyboard space | No hidden controls, scroll boundaries, sticky footers |
| Authentication | OAuth flow, token state, expired sessions, denied scopes | Signed out, expired token, denied access, success |
You do not need the same depth for every surface. A static read-only card needs less coverage than a multi-step editor with upload, save, and follow-up message actions.
Separate the Portable Layer from Host Extensions
Cross-host apps are easier to test when the architecture is clear:
- The portable layer uses MCP tools, resources,
content,structuredContent, resource metadata, host CSS variables, and standard app-to-host bridge actions. In sunpeak, helpers such asSafeArea,useToolData,useAppState,useDisplayMode,useHostContext,useCallServerTool, anduseSendMessagewrap common view behavior. - The host extension layer uses features that only exist in a target host, such as ChatGPT file library APIs, ChatGPT modal APIs, checkout, or Claude-specific connector behavior.
- The fallback layer explains what happens when the host does not support the extension.
That separation gives your tests something crisp to assert. Portable tests should pass on every host. Host extension tests should prove both the native path and the fallback.
Setting Up Cross-Host Tests
In a sunpeak project, cross-host coverage starts in Playwright config:
// playwright.config.ts
import { defineConfig } from 'sunpeak/test/config';
export default defineConfig();
defineConfig() creates host projects for the replicated ChatGPT and Claude runtimes. Each test runs once per configured host, and the report shows which host failed. The sunpeak E2E guide documents the fixture and project setup:
✓ [chatgpt] dashboard renders summary cards
✓ [chatgpt] dashboard adapts to dark mode
✗ [claude] dashboard renders summary cards
✓ [claude] dashboard adapts to dark mode
The test itself stays host-agnostic unless it is testing a host-specific branch:
import { expect, test } from 'sunpeak/test';
test('weather card shows the forecast', async ({ inspector }) => {
// Omit input to use a simulation fixture when this resource has one.
const result = await inspector.renderTool('show-weather');
const app = result.app();
await expect(app.getByRole('heading', { name: 'Portland forecast' })).toBeVisible();
await expect(app.getByText('82 degrees')).toBeVisible();
});
This catches missing text, broken tool output wiring, inaccessible headings, and host iframe load failures. Passing an input as the second argument calls the real server; omitting it uses a simulation fixture when one is available and otherwise calls the server with empty arguments. Keep the broad host matrix on pinned simulations, then add a smaller real-server suite for the tool contract.
Build a Compatibility Test Matrix
A useful matrix is small enough to run on every pull request and wide enough to catch the combinations that break real apps.
Start with these dimensions:
- Host: ChatGPT and Claude.
- Display mode: inline and fullscreen for most apps, picture-in-picture when you request it or support compact floating UI.
- Theme: light and dark.
- Data state: loading, success, empty, error, cancelled, and long content.
- Capability state: supported, unsupported, denied, and rejected.
- Viewport: at least one narrow width and one roomy fullscreen width for layout-sensitive resources.
Then write one matrix test around the riskiest resource, not every resource:
import { expect, test } from 'sunpeak/test';
const displayModes = ['inline', 'fullscreen'] as const;
const themes = ['light', 'dark'] as const;
for (const displayMode of displayModes) {
for (const theme of themes) {
test(`dashboard works in ${displayMode} ${theme}`, async ({ inspector }) => {
const result = await inspector.renderTool(
'show-dashboard',
undefined,
{ displayMode, theme },
);
const app = result.app();
await expect(app.getByRole('heading', { name: 'Revenue dashboard' })).toBeVisible();
await expect(app.getByRole('button', { name: 'Refresh' })).toBeEnabled();
await expect(app.locator('[data-testid="dashboard-root"]')).not.toHaveCSS(
'overflow-x',
'scroll',
);
});
}
}
Because the config already runs this per host, this loop produces 8 runs: 2 hosts, 2 display modes, and 2 themes. Only use modes the target host supports. For example, sunpeak’s E2E docs show a host-aware skip for Claude picture-in-picture coverage.
Test Tool Data Timing, Not Just Happy Paths
Host bridge timing is one of the easiest things to miss. In some flows, the iframe can mount before the final tool result is available. Approval-gated tools, auth handoffs, slow tools, and interrupted calls can all create states that a local happy-path render never sees.
Test these states directly:
import { expect, test } from 'sunpeak/test';
test('invoice app handles an empty result', async ({ inspector }) => {
// The test server returns an empty list for this stable customer ID.
const result = await inspector.renderTool('list-invoices', { customerId: 'cus_empty' });
await expect(result.app().getByText('No invoices found')).toBeVisible();
});
test('invoice app handles a tool error', async ({ inspector }) => {
// The test server returns a controlled error for this stable customer ID.
const result = await inspector.renderTool('list-invoices', { customerId: 'cus_error' });
await expect(result.app().getByRole('alert')).toContainText('Could not load invoices');
});
These tests should run across hosts. The snippets assume a reachable test server with stable customer IDs that produce the named states; adjust them to your server’s test data. If you use simulations instead, pin separate empty and error payloads through your project’s fixture setup. If an error state is clipped only in Claude inline mode, keep that regression in CI.
Test Feature Detection and Fallbacks
Host-specific APIs are useful when available. A host name only tells you where the app is running; it does not prove that a particular account or rollout has a feature. Detect the method itself and check the host response.
A payment sheet is a good example because OpenAI currently describes requestCheckout as a private beta for select marketplaces. For eligible integrations, the server returns a checkout session in structuredContent; the view calls window.openai.requestCheckout(session) after the user confirms. External checkout remains the generally available path. A capability check can choose which control to render:
import { useState } from 'react';
type Props = { checkoutUrl: string; checkoutSession?: unknown };
export function BuyButton({ checkoutUrl, checkoutSession }: Props) {
const [error, setError] = useState(false);
const bridge = typeof window === 'undefined' ? undefined : window as Window & {
openai?: { requestCheckout?: (session: unknown) => Promise<unknown> };
};
const requestCheckout = bridge.openai?.requestCheckout;
if (requestCheckout && checkoutSession) {
return <>
<button type="button" onClick={async () => {
try {
await requestCheckout(checkoutSession);
} catch {
setError(true);
}
}}>Buy now</button>
{error && <p role="alert">Checkout did not finish. Try again or use the web checkout.</p>}
<a href={checkoutUrl}>Buy on the web</a>
</>;
}
return <a href={checkoutUrl}>Buy on the web</a>;
}
Props here represents your app’s checkout URL and server-issued session type. Use your API’s concrete checkout-session type in production, and distinguish cancellation from payment errors. The server remains the source of truth for prices and order state. Keep the web path visible on both hosts:
import { expect, test } from 'sunpeak/test';
test('checkout offers a web path on both hosts', async ({ inspector }) => {
const result = await inspector.renderTool('show-product');
const app = result.app();
const external = app.getByRole('link', { name: 'Buy on the web' });
await expect(external).toBeVisible();
await expect(external).toHaveAttribute('href', /checkout/);
});
Add a focused test with a simulated bridge that supplies or removes requestCheckout to prove both branches deterministically. In a live ChatGPT account enabled for the payment sheet, also test a payment_mode: "test" session and a rejected payment. Use the same capability pattern for file pickers and modal APIs. If a feature has no safe fallback, test that the control is hidden or disabled with clear text.
Test Display Modes as Layout Contracts
Display modes are not just cosmetic. They change the job your layout has to do.
| Mode | Common failure | Better assertion |
|---|---|---|
| Inline | Content overflows horizontally or pushes controls below the fold | Root has no horizontal scroll, primary action remains visible |
| Fullscreen | Compact layout wastes space or sticky panels detach | Main content uses available width, navigation remains reachable |
| Picture-in-picture | Dense UI becomes unusable or host falls back to fullscreen | Compact controls render, unsupported modes degrade cleanly |
Write tests against user outcomes rather than exact pixel dimensions:
test('search results stay usable inline', async ({ inspector }) => {
const result = await inspector.renderTool(
'search-docs',
undefined,
{ displayMode: 'inline' },
);
const app = result.app();
await expect(app.getByRole('searchbox')).toBeVisible();
await expect(app.getByRole('button', { name: 'Filter' })).toBeVisible();
await expect(app.locator('[data-testid="results-panel"]')).not.toHaveCSS('overflow-x', 'scroll');
});
If you request display mode transitions, also test rejection. Hosts can deny a request, ignore unsupported modes, or use a fallback mode. Your app should still be usable.
Test Safe Areas and Host Chrome
Safe areas are where cross-host bugs hide. A sticky footer that works in your browser can sit under the host composer. A top toolbar can overlap the host header. A modal trigger can be visible in fullscreen and clipped inline.
Use SafeArea for the main resource shell, then assert the interactive controls are visible in the app iframe:
test('primary actions stay inside the safe area', async ({ inspector }) => {
const result = await inspector.renderTool(
'edit-report',
undefined,
{ displayMode: 'fullscreen', theme: 'dark' },
);
const app = result.app();
await expect(app.getByRole('button', { name: 'Save changes' })).toBeInViewport();
await expect(app.getByRole('button', { name: 'Send summary' })).toBeInViewport();
});
Safe-area tests are especially useful for resources with sticky headers, sticky footers, tabs, split panes, long lists, and mobile layouts.
Use Visual Regression Per Host
Structural assertions catch missing elements. They do not catch a link that has low contrast in Claude dark mode, a card border that disappears in ChatGPT, or a grid that shifts 12px after a CSS change.
For cross-host compatibility, keep screenshot baselines per host:
__screenshots__/
chatgpt/
dashboard-inline-light.png
dashboard-inline-dark.png
dashboard-fullscreen-dark.png
claude/
dashboard-inline-light.png
dashboard-inline-dark.png
dashboard-fullscreen-dark.png
The screenshots should not be identical. The point is for each host to look native and stay stable over time.
import { test } from 'sunpeak/test';
test('dashboard visual baseline', async ({ inspector }) => {
const result = await inspector.renderTool(
'show-dashboard',
undefined,
{ displayMode: 'fullscreen', theme: 'dark' },
);
await result.screenshot();
});
Run visual tests in CI when UI code changes:
pnpm test:visual
When the change is intentional, update baselines explicitly:
pnpm test:visual --update
Review the images before committing. The host name in the baseline path makes it much faster to spot whether a change affected every host or only one.
Add Targeted CSS Assertions
Visual diffs are great for product-level layout changes. CSS assertions are better when you want to prevent a specific class of bug.
Examples worth testing:
- A root container has no horizontal overflow in inline mode.
- A sticky footer remains inside the viewport.
- Links and buttons have visible focus states.
- Text meets the contrast target for its size in dark mode.
- The app uses host CSS variables instead of a hardcoded single-host palette.
test('dark mode links remain readable', async ({ inspector }) => {
const result = await inspector.renderTool(
'show-settings',
undefined,
{ theme: 'dark', displayMode: 'inline' },
);
const app = result.app();
const link = app.getByRole('link').first();
const contrast = await link.evaluate((el, panelSelector) => {
const panel = el.closest(panelSelector);
if (!panel) throw new Error('Settings panel missing');
const rgb = (value: string) => value.match(/\d+(?:\.\d+)?/g)!.slice(0, 3).map(Number);
const luminance = (value: string) => {
const channels = rgb(value).map((n) => {
const c = n / 255;
return c <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4;
});
return channels[0] * 0.2126 + channels[1] * 0.7152 + channels[2] * 0.0722;
};
const a = luminance(getComputedStyle(el).color);
const b = luminance(getComputedStyle(panel).backgroundColor);
return (Math.max(a, b) + 0.05) / (Math.min(a, b) + 0.05);
}, '[data-testid="settings-panel"]');
expect(contrast).toBeGreaterThanOrEqual(4.5);
});
The WCAG contrast minimum is 4.5:1 for normal text. This example assumes an opaque panel background and a solid link color. If your UI uses transparency, gradients, or layered backgrounds, measure the final rendered colors or use an accessibility checker that accounts for those layers.
Run the Matrix in CI
Your CI job should run deterministic local tests before any live-host smoke tests. A good default is:
name: Test
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: pnpm
- run: pnpm install
- run: pnpm exec playwright install --with-deps chromium
- run: pnpm test
- run: pnpm test:visual
- uses: actions/upload-artifact@v4
if: failure()
with:
name: test-results
path: test-results/
Keep live tests separate. Local inspector tests are deterministic and should run on every pull request. Live tests against real hosts are useful before release, but they depend on real accounts, host availability, auth state, and network timing.
A Cross-Host Release Checklist
Use this checklist before publishing a new resource or changing a shared component:
- Protocol contract: tool input, output schema,
structuredContent,_meta, resource URI, and resource metadata are covered by integration tests. - Rendering states: loading, success, empty, error, cancelled, and long-content states render in the inspector.
- Host matrix: ChatGPT and Claude projects both pass for the main resource.
- Display modes: inline and fullscreen pass, and PiP passes or degrades cleanly where relevant.
- Themes: light and dark visual baselines exist for layout-sensitive resources.
- Safe area: primary actions stay visible inside host chrome.
- Feature detection: every host-only branch has a tested fallback.
- Auth and permissions: signed-out, denied, expired, and success states are covered where the tool needs auth.
- Live smoke: before release, a tiny live suite proves discovery, tool invocation, resource render, and the main user workflow in each target host.
If you use the sunpeak testing framework, most of this can run locally and in CI without paid host accounts or AI credits. npx sunpeak inspect --server URL is useful for manual checks, and the Playwright fixtures let you turn the same host states into repeatable tests.
When Not to Add More Matrix Tests
Cross-host coverage should be focused. Do not write 12 combinations for every paragraph, badge, or static read-only component.
Use the full matrix for:
- Layout-sensitive resources with grids, charts, split panes, sticky controls, or dense tables.
- Resources with host-specific actions such as upload, checkout, modal, external link, or display mode requests.
- Resources with auth state, long content, or tool calls from the UI.
- Shared shell components used by many resources.
Use simpler smoke coverage for:
- Static summary cards.
- Plain text result views.
- Components already covered through a higher-level resource test.
The goal is not maximal combinations. The goal is catching the bugs that only appear when the same MCP App runs inside a different host.
Get Started
npx sunpeak newFurther Reading
- E2E testing MCP Apps - the inspector fixture, renderTool, and cross-host projects
- Build one MCP App for ChatGPT and Claude - the cross-host architecture
- MCP App capability detection and fallbacks
- Visual regression testing MCP Apps - screenshot baselines per host
- MCP App styling - host CSS variables, dark mode, and native-looking UIs
- MCP App actions - callServerTool, sendMessage, and updateModelContext
- Live testing Claude Connectors and ChatGPT Apps
- sunpeak testing framework
- sunpeak MCP App framework
- Official MCP Apps overview and capability negotiation
- OpenAI guide to MCP Apps UI and optional ChatGPT extensions
- Anthropic guide to building Claude connectors
Frequently Asked Questions
Why do I need cross-host testing for MCP Apps?
MCP Apps render inside host-controlled iframes, and each host can differ in CSS variables, viewport size, safe areas, display mode support, bridge timing, file APIs, and host-specific actions. Cross-host testing verifies the portable MCP App path works everywhere and that ChatGPT-only or Claude-only features have tested fallbacks.
What should I test across ChatGPT and Claude?
Test the protocol contract first: tool input, structuredContent, resource metadata, loading states, error states, and app-to-host actions. Then test host-dependent UI: themes, display modes, safe areas, narrow inline layouts, fullscreen layouts, file flows, modals, checkout, authentication, and any feature detection branch that changes what the user sees.
How do I run MCP App tests against both ChatGPT and Claude?
In a sunpeak project, use defineConfig() from sunpeak/test/config in your Playwright config. It creates host projects for the replicated ChatGPT and Claude runtimes, so the same test file runs per host. Use inspector.renderTool() to render a tool result, then assert against result.app() with Playwright.
How do I test host-specific feature detection in MCP Apps?
Use shared MCP App bridge methods for portable behavior. Check whether an optional method exists before using it, then test the supported and fallback paths. For example, check for window.openai.requestCheckout before offering an eligible ChatGPT payment sheet; otherwise link to external checkout. Do not infer capabilities from a host name.
What is a compatibility test matrix for MCP Apps?
A compatibility test matrix is the small set of host, display mode, theme, viewport, data, and capability combinations that can realistically break your app. For many apps, that means ChatGPT and Claude, inline and fullscreen, light and dark themes, one empty state, one error state, one long-content state, and each host-specific feature branch.
How do I test CSS differences between MCP App hosts?
Use host CSS variables instead of hardcoded colors, then run visual regression tests per host and theme. For targeted checks, use Playwright to assert computed styles, contrast, overflow, and safe-area spacing. Keep separate screenshot baselines for ChatGPT and Claude because the hosts should look native rather than identical.
Can I skip a test on a specific MCP App host?
Yes. Use host-aware skips only when the behavior is truly host-specific, such as a ChatGPT-only file picker or a display mode a host does not support. Avoid skipping portable protocol behavior because those tests are the ones that prove your MCP App can run across hosts.
How do I test MCP Apps for hosts beyond ChatGPT and Claude?
Keep the app on MCP Apps tools, resources, structuredContent, host notifications, and app-to-host actions. Check whether the new host advertises UI support and make tool content readable when it does not. Add a live smoke suite for discovery, tool invocation, resource rendering where supported, theme, and the main user workflow.