Skip to main content
All posts

Cross-Host Compatibility Testing for MCP Apps: ChatGPT, Claude, and Beyond (October 2026)

Abe Wheeler
MCP AppsMCP App TestingMCP App FrameworkChatGPT AppsChatGPT App TestingChatGPT App FrameworkClaude ConnectorsClaude Connector TestingClaude Connector FrameworkCross-Host TestingCompatibility Testing
Cross-host compatibility testing catches bugs that only appear in specific MCP App hosts.

Cross-host compatibility testing catches bugs that only appear in specific MCP App hosts.

Your MCP App passes every test. Green across the board. You publish it for ChatGPT, someone installs the same server as a Claude Connector, and a card that looked polished in one host clips in the other. The tool call works, but the action button does nothing. Dark mode passes in your browser and fails inside the host iframe.

That is the core cross-host problem. MCP Apps are designed to be portable, but the app does not run in a browser you control. It runs inside a host-controlled iframe, with host-owned chrome, safe areas, display modes, CSS variables, bridge timing, and optional platform APIs.

TL;DR: Start with the MCP tool contract and a readable text response for hosts without UI support. Run one focused Playwright matrix across ChatGPT and Claude for display modes, themes, data states, and capability fallbacks. Use standard MCP Apps bridge methods for shared behavior, feature-detect optional host APIs, and keep separate visual baselines per host.

What Changed Since the First Wave of ChatGPT Apps

The official MCP Apps extension defines a portable pattern: a tool declares a ui:// resource in _meta.ui.resourceUri, the host calls the tool and fetches that resource, and the UI renders in a sandboxed iframe. The UI receives tool input and results through the host bridge, then sends actions back through that bridge.

OpenAI’s UI guide recommends shared MCP Apps fields and bridge methods for new UI. ChatGPT also exposes optional window.openai extensions for file handling, modals, and eligible checkout flows. OpenAI recommends detecting each extension and providing a fallback where practical, rather than branching on the product name.

Anthropic’s Claude connector guide covers remote MCP servers and optional MCP Apps UI. The same server may be called by a host that can render the UI and one that only consumes the tool response. Test both paths when you claim broad host compatibility.

That gives you two contracts to verify: the server response must stand on its own, and the interactive view must work when a host supports it.

Check the Protocol Before the Pixels

MCP Apps is an extension that hosts advertise during capability negotiation. A server can register UI-enabled tools for supporting hosts and still provide ordinary tools to hosts without UI support. In the UI path, content remains readable to the model, while structuredContent can provide data for the view. If a screenshot looks right but the text response is an empty placeholder, a text-only host has little to work with.

Before opening a browser, inspect one representative tool response and answer these questions:

  1. Does the tool have a stable input and output schema, and does _meta.ui.resourceUri point to a resource the host can fetch?
  2. Can someone understand the result from content alone when no iframe is rendered?
  3. Does structuredContent contain the fields the view needs, including explicit empty and error states?
  4. Are UI-only actions exposed as app-visible tools where appropriate, and can the host reject them without corrupting the view?

The MCP Apps overview explains the tool-to-resource link and the text fallback. The official testing guide describes a reference host you can use to inspect a server outside your target product.

What Differs Between Hosts

Before writing tests, list the surfaces where a host can change behavior. The tool result can be valid and the UI can still fail because one of these surfaces changed.

SurfaceWhat can differWhat to test
Resource iframeSandbox, origin, allowed network targets, iframe sizingResource loads, CSP works, external assets render
Tool dataTiming of input/result delivery, _meta, structuredContent, empty statesLoading, approval, success, error, cancelled, empty
Host bridgecallServerTool, messages, model context, optional platform APIsAccepted path, rejected path, missing capability path
ThemeCSS variables, dark palette, font stack, contrastLight, dark, computed styles, screenshot baselines
Display modeInline, fullscreen, picture-in-picture, unsupported modesLayout, overflow, mode request fallback
Safe areaHeader, sidebar, mobile chrome, keyboard spaceNo hidden controls, scroll boundaries, sticky footers
AuthenticationOAuth flow, token state, expired sessions, denied scopesSigned out, expired token, denied access, success

You do not need the same depth for every surface. A static read-only card needs less coverage than a multi-step editor with upload, save, and follow-up message actions.

Separate the Portable Layer from Host Extensions

Cross-host apps are easier to test when the architecture is clear:

  • The portable layer uses MCP tools, resources, content, structuredContent, resource metadata, host CSS variables, and standard app-to-host bridge actions. In sunpeak, helpers such as SafeArea, useToolData, useAppState, useDisplayMode, useHostContext, useCallServerTool, and useSendMessage wrap common view behavior.
  • The host extension layer uses features that only exist in a target host, such as ChatGPT file library APIs, ChatGPT modal APIs, checkout, or Claude-specific connector behavior.
  • The fallback layer explains what happens when the host does not support the extension.

That separation gives your tests something crisp to assert. Portable tests should pass on every host. Host extension tests should prove both the native path and the fallback.

Setting Up Cross-Host Tests

In a sunpeak project, cross-host coverage starts in Playwright config:

// playwright.config.ts
import { defineConfig } from 'sunpeak/test/config';

export default defineConfig();

defineConfig() creates host projects for the replicated ChatGPT and Claude runtimes. Each test runs once per configured host, and the report shows which host failed. The sunpeak E2E guide documents the fixture and project setup:

✓ [chatgpt] dashboard renders summary cards
✓ [chatgpt] dashboard adapts to dark mode
✗ [claude] dashboard renders summary cards
✓ [claude] dashboard adapts to dark mode

The test itself stays host-agnostic unless it is testing a host-specific branch:

import { expect, test } from 'sunpeak/test';

test('weather card shows the forecast', async ({ inspector }) => {
  // Omit input to use a simulation fixture when this resource has one.
  const result = await inspector.renderTool('show-weather');

  const app = result.app();

  await expect(app.getByRole('heading', { name: 'Portland forecast' })).toBeVisible();
  await expect(app.getByText('82 degrees')).toBeVisible();
});

This catches missing text, broken tool output wiring, inaccessible headings, and host iframe load failures. Passing an input as the second argument calls the real server; omitting it uses a simulation fixture when one is available and otherwise calls the server with empty arguments. Keep the broad host matrix on pinned simulations, then add a smaller real-server suite for the tool contract.

Build a Compatibility Test Matrix

A useful matrix is small enough to run on every pull request and wide enough to catch the combinations that break real apps.

Start with these dimensions:

  • Host: ChatGPT and Claude.
  • Display mode: inline and fullscreen for most apps, picture-in-picture when you request it or support compact floating UI.
  • Theme: light and dark.
  • Data state: loading, success, empty, error, cancelled, and long content.
  • Capability state: supported, unsupported, denied, and rejected.
  • Viewport: at least one narrow width and one roomy fullscreen width for layout-sensitive resources.

Then write one matrix test around the riskiest resource, not every resource:

import { expect, test } from 'sunpeak/test';

const displayModes = ['inline', 'fullscreen'] as const;
const themes = ['light', 'dark'] as const;

for (const displayMode of displayModes) {
  for (const theme of themes) {
    test(`dashboard works in ${displayMode} ${theme}`, async ({ inspector }) => {
      const result = await inspector.renderTool(
        'show-dashboard',
        undefined,
        { displayMode, theme },
      );

      const app = result.app();

      await expect(app.getByRole('heading', { name: 'Revenue dashboard' })).toBeVisible();
      await expect(app.getByRole('button', { name: 'Refresh' })).toBeEnabled();
      await expect(app.locator('[data-testid="dashboard-root"]')).not.toHaveCSS(
        'overflow-x',
        'scroll',
      );
    });
  }
}

Because the config already runs this per host, this loop produces 8 runs: 2 hosts, 2 display modes, and 2 themes. Only use modes the target host supports. For example, sunpeak’s E2E docs show a host-aware skip for Claude picture-in-picture coverage.

Test Tool Data Timing, Not Just Happy Paths

Host bridge timing is one of the easiest things to miss. In some flows, the iframe can mount before the final tool result is available. Approval-gated tools, auth handoffs, slow tools, and interrupted calls can all create states that a local happy-path render never sees.

Test these states directly:

import { expect, test } from 'sunpeak/test';

test('invoice app handles an empty result', async ({ inspector }) => {
  // The test server returns an empty list for this stable customer ID.
  const result = await inspector.renderTool('list-invoices', { customerId: 'cus_empty' });

  await expect(result.app().getByText('No invoices found')).toBeVisible();
});

test('invoice app handles a tool error', async ({ inspector }) => {
  // The test server returns a controlled error for this stable customer ID.
  const result = await inspector.renderTool('list-invoices', { customerId: 'cus_error' });

  await expect(result.app().getByRole('alert')).toContainText('Could not load invoices');
});

These tests should run across hosts. The snippets assume a reachable test server with stable customer IDs that produce the named states; adjust them to your server’s test data. If you use simulations instead, pin separate empty and error payloads through your project’s fixture setup. If an error state is clipped only in Claude inline mode, keep that regression in CI.

Test Feature Detection and Fallbacks

Host-specific APIs are useful when available. A host name only tells you where the app is running; it does not prove that a particular account or rollout has a feature. Detect the method itself and check the host response.

A payment sheet is a good example because OpenAI currently describes requestCheckout as a private beta for select marketplaces. For eligible integrations, the server returns a checkout session in structuredContent; the view calls window.openai.requestCheckout(session) after the user confirms. External checkout remains the generally available path. A capability check can choose which control to render:

import { useState } from 'react';

type Props = { checkoutUrl: string; checkoutSession?: unknown };

export function BuyButton({ checkoutUrl, checkoutSession }: Props) {
  const [error, setError] = useState(false);
  const bridge = typeof window === 'undefined' ? undefined : window as Window & {
    openai?: { requestCheckout?: (session: unknown) => Promise<unknown> };
  };
  const requestCheckout = bridge.openai?.requestCheckout;

  if (requestCheckout && checkoutSession) {
    return <>
      <button type="button" onClick={async () => {
        try {
          await requestCheckout(checkoutSession);
        } catch {
          setError(true);
        }
      }}>Buy now</button>
      {error && <p role="alert">Checkout did not finish. Try again or use the web checkout.</p>}
      <a href={checkoutUrl}>Buy on the web</a>
    </>;
  }

  return <a href={checkoutUrl}>Buy on the web</a>;
}

Props here represents your app’s checkout URL and server-issued session type. Use your API’s concrete checkout-session type in production, and distinguish cancellation from payment errors. The server remains the source of truth for prices and order state. Keep the web path visible on both hosts:

import { expect, test } from 'sunpeak/test';

test('checkout offers a web path on both hosts', async ({ inspector }) => {
  const result = await inspector.renderTool('show-product');
  const app = result.app();

  const external = app.getByRole('link', { name: 'Buy on the web' });
  await expect(external).toBeVisible();
  await expect(external).toHaveAttribute('href', /checkout/);
});

Add a focused test with a simulated bridge that supplies or removes requestCheckout to prove both branches deterministically. In a live ChatGPT account enabled for the payment sheet, also test a payment_mode: "test" session and a rejected payment. Use the same capability pattern for file pickers and modal APIs. If a feature has no safe fallback, test that the control is hidden or disabled with clear text.

Test Display Modes as Layout Contracts

Display modes are not just cosmetic. They change the job your layout has to do.

ModeCommon failureBetter assertion
InlineContent overflows horizontally or pushes controls below the foldRoot has no horizontal scroll, primary action remains visible
FullscreenCompact layout wastes space or sticky panels detachMain content uses available width, navigation remains reachable
Picture-in-pictureDense UI becomes unusable or host falls back to fullscreenCompact controls render, unsupported modes degrade cleanly

Write tests against user outcomes rather than exact pixel dimensions:

test('search results stay usable inline', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'search-docs',
    undefined,
    { displayMode: 'inline' },
  );

  const app = result.app();

  await expect(app.getByRole('searchbox')).toBeVisible();
  await expect(app.getByRole('button', { name: 'Filter' })).toBeVisible();
  await expect(app.locator('[data-testid="results-panel"]')).not.toHaveCSS('overflow-x', 'scroll');
});

If you request display mode transitions, also test rejection. Hosts can deny a request, ignore unsupported modes, or use a fallback mode. Your app should still be usable.

Test Safe Areas and Host Chrome

Safe areas are where cross-host bugs hide. A sticky footer that works in your browser can sit under the host composer. A top toolbar can overlap the host header. A modal trigger can be visible in fullscreen and clipped inline.

Use SafeArea for the main resource shell, then assert the interactive controls are visible in the app iframe:

test('primary actions stay inside the safe area', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'edit-report',
    undefined,
    { displayMode: 'fullscreen', theme: 'dark' },
  );

  const app = result.app();

  await expect(app.getByRole('button', { name: 'Save changes' })).toBeInViewport();
  await expect(app.getByRole('button', { name: 'Send summary' })).toBeInViewport();
});

Safe-area tests are especially useful for resources with sticky headers, sticky footers, tabs, split panes, long lists, and mobile layouts.

Use Visual Regression Per Host

Structural assertions catch missing elements. They do not catch a link that has low contrast in Claude dark mode, a card border that disappears in ChatGPT, or a grid that shifts 12px after a CSS change.

For cross-host compatibility, keep screenshot baselines per host:

__screenshots__/
  chatgpt/
    dashboard-inline-light.png
    dashboard-inline-dark.png
    dashboard-fullscreen-dark.png
  claude/
    dashboard-inline-light.png
    dashboard-inline-dark.png
    dashboard-fullscreen-dark.png

The screenshots should not be identical. The point is for each host to look native and stay stable over time.

import { test } from 'sunpeak/test';

test('dashboard visual baseline', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-dashboard',
    undefined,
    { displayMode: 'fullscreen', theme: 'dark' },
  );

  await result.screenshot();
});

Run visual tests in CI when UI code changes:

pnpm test:visual

When the change is intentional, update baselines explicitly:

pnpm test:visual --update

Review the images before committing. The host name in the baseline path makes it much faster to spot whether a change affected every host or only one.

Add Targeted CSS Assertions

Visual diffs are great for product-level layout changes. CSS assertions are better when you want to prevent a specific class of bug.

Examples worth testing:

  • A root container has no horizontal overflow in inline mode.
  • A sticky footer remains inside the viewport.
  • Links and buttons have visible focus states.
  • Text meets the contrast target for its size in dark mode.
  • The app uses host CSS variables instead of a hardcoded single-host palette.
test('dark mode links remain readable', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-settings',
    undefined,
    { theme: 'dark', displayMode: 'inline' },
  );

  const app = result.app();
  const link = app.getByRole('link').first();

  const contrast = await link.evaluate((el, panelSelector) => {
    const panel = el.closest(panelSelector);
    if (!panel) throw new Error('Settings panel missing');
    const rgb = (value: string) => value.match(/\d+(?:\.\d+)?/g)!.slice(0, 3).map(Number);
    const luminance = (value: string) => {
      const channels = rgb(value).map((n) => {
        const c = n / 255;
        return c <= 0.04045 ? c / 12.92 : ((c + 0.055) / 1.055) ** 2.4;
      });
      return channels[0] * 0.2126 + channels[1] * 0.7152 + channels[2] * 0.0722;
    };
    const a = luminance(getComputedStyle(el).color);
    const b = luminance(getComputedStyle(panel).backgroundColor);
    return (Math.max(a, b) + 0.05) / (Math.min(a, b) + 0.05);
  }, '[data-testid="settings-panel"]');

  expect(contrast).toBeGreaterThanOrEqual(4.5);
});

The WCAG contrast minimum is 4.5:1 for normal text. This example assumes an opaque panel background and a solid link color. If your UI uses transparency, gradients, or layered backgrounds, measure the final rendered colors or use an accessibility checker that accounts for those layers.

Run the Matrix in CI

Your CI job should run deterministic local tests before any live-host smoke tests. A good default is:

name: Test
on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: pnpm
      - run: pnpm install
      - run: pnpm exec playwright install --with-deps chromium
      - run: pnpm test
      - run: pnpm test:visual
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: test-results
          path: test-results/

Keep live tests separate. Local inspector tests are deterministic and should run on every pull request. Live tests against real hosts are useful before release, but they depend on real accounts, host availability, auth state, and network timing.

A Cross-Host Release Checklist

Use this checklist before publishing a new resource or changing a shared component:

  1. Protocol contract: tool input, output schema, structuredContent, _meta, resource URI, and resource metadata are covered by integration tests.
  2. Rendering states: loading, success, empty, error, cancelled, and long-content states render in the inspector.
  3. Host matrix: ChatGPT and Claude projects both pass for the main resource.
  4. Display modes: inline and fullscreen pass, and PiP passes or degrades cleanly where relevant.
  5. Themes: light and dark visual baselines exist for layout-sensitive resources.
  6. Safe area: primary actions stay visible inside host chrome.
  7. Feature detection: every host-only branch has a tested fallback.
  8. Auth and permissions: signed-out, denied, expired, and success states are covered where the tool needs auth.
  9. Live smoke: before release, a tiny live suite proves discovery, tool invocation, resource render, and the main user workflow in each target host.

If you use the sunpeak testing framework, most of this can run locally and in CI without paid host accounts or AI credits. npx sunpeak inspect --server URL is useful for manual checks, and the Playwright fixtures let you turn the same host states into repeatable tests.

When Not to Add More Matrix Tests

Cross-host coverage should be focused. Do not write 12 combinations for every paragraph, badge, or static read-only component.

Use the full matrix for:

  • Layout-sensitive resources with grids, charts, split panes, sticky controls, or dense tables.
  • Resources with host-specific actions such as upload, checkout, modal, external link, or display mode requests.
  • Resources with auth state, long content, or tool calls from the UI.
  • Shared shell components used by many resources.

Use simpler smoke coverage for:

  • Static summary cards.
  • Plain text result views.
  • Components already covered through a higher-level resource test.

The goal is not maximal combinations. The goal is catching the bugs that only appear when the same MCP App runs inside a different host.

Get Started

Documentation →
npx sunpeak new

Further Reading

Frequently Asked Questions

Why do I need cross-host testing for MCP Apps?

MCP Apps render inside host-controlled iframes, and each host can differ in CSS variables, viewport size, safe areas, display mode support, bridge timing, file APIs, and host-specific actions. Cross-host testing verifies the portable MCP App path works everywhere and that ChatGPT-only or Claude-only features have tested fallbacks.

What should I test across ChatGPT and Claude?

Test the protocol contract first: tool input, structuredContent, resource metadata, loading states, error states, and app-to-host actions. Then test host-dependent UI: themes, display modes, safe areas, narrow inline layouts, fullscreen layouts, file flows, modals, checkout, authentication, and any feature detection branch that changes what the user sees.

How do I run MCP App tests against both ChatGPT and Claude?

In a sunpeak project, use defineConfig() from sunpeak/test/config in your Playwright config. It creates host projects for the replicated ChatGPT and Claude runtimes, so the same test file runs per host. Use inspector.renderTool() to render a tool result, then assert against result.app() with Playwright.

How do I test host-specific feature detection in MCP Apps?

Use shared MCP App bridge methods for portable behavior. Check whether an optional method exists before using it, then test the supported and fallback paths. For example, check for window.openai.requestCheckout before offering an eligible ChatGPT payment sheet; otherwise link to external checkout. Do not infer capabilities from a host name.

What is a compatibility test matrix for MCP Apps?

A compatibility test matrix is the small set of host, display mode, theme, viewport, data, and capability combinations that can realistically break your app. For many apps, that means ChatGPT and Claude, inline and fullscreen, light and dark themes, one empty state, one error state, one long-content state, and each host-specific feature branch.

How do I test CSS differences between MCP App hosts?

Use host CSS variables instead of hardcoded colors, then run visual regression tests per host and theme. For targeted checks, use Playwright to assert computed styles, contrast, overflow, and safe-area spacing. Keep separate screenshot baselines for ChatGPT and Claude because the hosts should look native rather than identical.

Can I skip a test on a specific MCP App host?

Yes. Use host-aware skips only when the behavior is truly host-specific, such as a ChatGPT-only file picker or a display mode a host does not support. Avoid skipping portable protocol behavior because those tests are the ones that prove your MCP App can run across hosts.

How do I test MCP Apps for hosts beyond ChatGPT and Claude?

Keep the app on MCP Apps tools, resources, structuredContent, host notifications, and app-to-host actions. Check whether the new host advertises UI support and make tool content readable when it does not. Add a live smoke suite for discovery, tool invocation, resource rendering where supported, theme, and the main user workflow.