Visual Regression Testing for MCP Apps, ChatGPT Apps, and Claude Connectors (September 2026)

Your MCP App can pass unit tests, schema tests, and snapshot tests, then still look broken in the host. A gap change clips a button in picture-in-picture. A dark-mode token resolves differently in Claude. A ChatGPT display mode gives you less width than your local browser tab. The markup is valid, but the user sees a broken UI.

Visual regression testing catches that class of bug. It renders your app in a real browser, captures screenshots, and compares them to approved baselines. When something visible changes, you get a diff image instead of a vague “looks off” report.

TL;DR: Treat a visual baseline as a record of six inputs: host, theme, container, locale, data state, and production resource build. Add result.screenshot() only after the app reaches a named stable state, then run pnpm test:visual or npx sunpeak test --visual. Keep one baseline per host project, upload expected, actual, diff, and trace artifacts in CI, and update a baseline only after reviewing the pixels and the underlying network and console state. sunpeak 0.20.84 runs this loop against local ChatGPT and Claude host replicas, while a small live-host suite covers integration details that a replica cannot prove.

Why visual regression testing matters for MCP Apps

MCP Apps are embedded web apps. They render through resources, iframes, host bridge APIs, host CSS variables, display modes, and content security policy metadata. That makes them more sensitive to host context than a normal single-page app.

The visual surface can change because of:

• Your CSS, component library, or layout code
• Tool result data, including long text, empty states, and malformed fields
• Host theme variables in light and dark mode
• Display modes such as inline, fullscreen, and picture-in-picture
• Iframe sizing, safe areas, scroll containers, and host chrome
• Remote images, custom fonts, and CSP allowlists
• Browser and operating system rendering differences between local machines and CI

E2E tests tell you whether the app works. Visual tests tell you whether the app still looks like something a user can use.

The September 2026 testing contract

MCP Apps now have a stable extension contract dated 2026-01-26. A tool points to a ui:// resource through _meta.ui.resourceUri, the host renders that resource in a sandboxed iframe, and the View talks to the host through JSON-RPC over postMessage. The current MCP Apps overview also makes two visual inputs explicit: _meta.ui.csp controls the origins an app may load, and host context can supply theme, styles, container dimensions, safe-area insets, locale, platform, and display-mode data.

OpenAI now documents these experiences under Plugins. The portable UI contract still uses the MCP Apps ui/* bridge. ChatGPT-specific additions can be feature-detected when an app needs them. For visual testing, this means your baseline should prove the portable View first, then cover any host-only surface in that host’s project or live smoke suite.

Claude’s current interactive connectors can render MCP Apps as inline cards and fullscreen views across web, desktop, Cowork, iOS, and Android. A desktop-only screenshot therefore cannot stand in for the whole supported surface. Test container pressure and touch behavior locally, then verify a small set of real connector prompts on the host versions you ship to.

sunpeak 0.20.84 turns host state into repeatable Playwright inputs. Its generated config creates separate ChatGPT and Claude projects, inspector.renderTool() sets theme and display mode, simulations pin tool states, and result.screenshot() compares the rendered app body. This keeps shell changes in the local inspector out of app baselines.

Define the baseline key before writing tests

A screenshot name such as dashboard.png hides too much. A useful baseline has a key, even if part of that key lives in the directory structure. Record the host, browser and operating system, theme, container, locale, data state, and resource build.

For example:

• ChatGPT baseline: chromium-linux, dark theme, 390-pixel inline container, en-US, long labels, production resource
• Claude baseline: chromium-linux, light theme, fullscreen, de-DE, empty state, production resource

The key stops two common mistakes. First, a host-specific change cannot silently replace another host’s expected pixels. Second, reviewers can tell whether a diff comes from app code, input data, host context, or the rendering environment.

You do not need the full cross product. Pick pairs and edge cases that exercise each independent risk at least once. If both hosts use the same app code, test the main state in both, then put data extremes in one host unless you have evidence that the other host handles them differently.

How visual regression testing works

The workflow is simple:

1. Render an MCP resource through a tool result or simulation.
2. Wait for stable app state, including data, images, and fonts.
3. Capture a screenshot of the app iframe, the full host page, or a specific element.
4. Compare the image against a committed baseline.
5. Fail the test when the diff crosses the configured threshold.
6. Store the baseline, actual screenshot, diff image, and trace for review.

The first run creates baselines. Later runs compare against those baselines. When you make an intentional UI change, update the baseline and commit it with the code change.

Write your first visual regression test

If your project already uses the inspector fixture from sunpeak/test, visual testing is one extra line in an E2E test.

// tests/e2e/albums.visual.spec.ts
import { expect, test } from 'sunpeak/test';

test('albums resource renders the default state', async ({ inspector }) => {
  const result = await inspector.renderTool('show-albums', {
    artist: 'Radiohead',
  });

  const app = result.app();
  await expect(app.getByText('OK Computer')).toBeVisible();

  await result.screenshot('albums-default');
});

In a sunpeak app, run:

pnpm test:visual

For an existing MCP server in any language, scaffold a test project first:

npx sunpeak test init --server http://localhost:8000/mcp
npx sunpeak test --visual

Generate or refresh baselines only after you have reviewed the rendered screenshots:

npx sunpeak test --visual --update

Choose screenshot targets carefully

sunpeak screenshots the MCP App body by default. This is the right boundary for a portable baseline because the local inspector shell is test infrastructure, not product UI. Older examples used target: 'page'; current sunpeak ignores that target and warns because an inspector navigation change should not rewrite every app baseline.

Use a focused element when one component needs a stricter or more stable comparison. If you need to test the full conversation shell, do that in a separate real-host or host-owned suite where the shell itself is part of the product under test.

// App body, the default portable baseline
await result.screenshot('albums-default');

// Specific element, useful for focused component checks
await result.screenshot('save-button', {
  element: result.app().getByRole('button', { name: 'Save' }),
});

A good naming scheme includes the state, host, theme, and display mode when those are not already encoded by the test project:

albums-default-light-inline
albums-empty-dark-inline
albums-overflow-dark-pip
albums-error-light-fullscreen

Test the host matrix that can break

Do not screenshot every possible combination. Pick the combinations where the UI can realistically break and where a diff would change your release decision.

Hosts

Keep separate baselines for ChatGPT and Claude when you support both. The app code can be portable, but the host context, theme values, iframe constraints, and supported modes can differ. In sunpeak, the generated Playwright config creates one project per host, so {projectName} can separate baseline paths without putting the host in every screenshot name.

tests/__screenshots__/
  chatgpt/
    albums.visual.spec.ts/
      albums-default.png
  claude/
    albums.visual.spec.ts/
      albums-default.png

Separate baselines are better than a single “universal” baseline because they make host-specific regressions obvious.

Themes

Take dark-mode screenshots for any surface that uses host CSS variables, semantic colors, borders, shadows, or charts.

test('albums render in dark mode', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { theme: 'dark' },
  );

  await result.screenshot('albums-dark');
});

Dark mode catches problems that text assertions miss: invisible text, low-contrast dividers, hardcoded backgrounds, and icons that disappear against the host shell.

Display modes

Display modes change viewport constraints and scroll behavior. Inline mode is usually narrow, fullscreen exposes wide layouts, and picture-in-picture tends to reveal overflow when a host supports it. Read the host’s advertised modes instead of assuming every mode exists. Claude currently documents inline cards and fullscreen views, while other hosts may expose a different set.

test('albums handle picture-in-picture constraints', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { displayMode: 'pip' },
  );

  await result.screenshot('albums-pip');
});

If your app asks the host to change display mode through the bridge, test the result after the host has accepted or rejected that request. Hosts can choose how to handle the request, so your UI should still work when the host keeps the current mode. Also test a resize that arrives through a host-context change after initial render because stale dimensions often cause clipping only after a mode transition.

Container size, safe areas, and locale

Display mode names are not viewport sizes. Add explicit narrow and wide containers, then test nonzero safe-area insets for fixed buttons, bottom sheets, and edge-to-edge controls. CSS container queries should drive layout, while host context can supply the limits and safe-area values that CSS alone cannot describe.

Locale is a visual input too. Use at least one fixture with longer translated labels, a different date format, and numbers that group differently. You do not need screenshots for every locale. Pick a locale that makes the layout work harder, and keep the locale in the baseline key.

Build deterministic test states

Visual tests fail when the screenshot changes. That is the point, but it means volatile data has to be removed or controlled.

Use simulation files or fixtures for:

• Tool input and tool result data
• Empty states and error states
• User profile names, avatar URLs, and timestamps
• Server tool responses triggered by app actions
• Long labels, long numbers, and localized strings

Avoid live APIs in baseline tests. If you need a live API smoke test, keep it outside the visual baseline suite or mask the volatile region.

For an MCP App, freeze every data lane that reaches the View:

• Tool arguments, including defaults the server fills in
• structuredContent and model-visible content
• View-only result metadata in _meta
• Follow-up tools/call responses triggered by a click
• Host-context changes sent after initialization

Name the fixture after the user-visible state, such as orders.empty, orders.permission-denied, or orders.long-german-labels. A visual test should not have to explain how a magic account ID creates the state.

Run the main release baselines against built resources too. A dev resource can hide bundling, CSP, asset-path, and minification failures. sunpeak’s generated tests support prodResources: true in the render options, so keep fast development screenshots for iteration and add a smaller production-resource set before release.

Stabilize screenshots before comparing pixels

Most flaky visual tests come from timing, not from the diff engine. Before you capture, make sure the page is ready.

test('dashboard visual state is stable', async ({ inspector }) => {
  const result = await inspector.renderTool('show-dashboard', {
    accountId: 'acct_visual_fixture',
  });

  const app = result.app();
  await expect(app.getByRole('heading', { name: 'Pipeline' })).toBeVisible();
  await expect(app.getByTestId('loading-skeleton')).toBeHidden();

  await result.screenshot('dashboard-default', {
    animations: 'disabled',
    threshold: 0.2,
    maxDiffPixelRatio: 0.01,
  });
});

Use this checklist before adding a screenshot:

• Data is fixed and comes from a simulation or fixture.
• Animations, transitions, cursors, and skeletons are gone or disabled.
• Fonts have loaded.
• Images are local, mocked, or already loaded.
• Timestamps and relative dates are frozen or hidden.
• Viewport size and display mode are explicit.
• Scroll position is controlled.
• Host, theme, and locale are explicit when they affect rendering.

Treat console errors and failed app requests as failures before you approve the pixels. A baseline with a missing font, blocked image, or rejected bridge call can be perfectly repeatable and still be wrong. The visual assertion should be the last assertion in a state, after semantic checks prove that the intended content and controls appeared.

Configure thresholds with intent

Pixel-perfect comparisons sound clean, but they break too often across machines. Anti-aliasing, font hinting, browser versions, and GPU differences can produce harmless pixel changes.

Start strict, then loosen only where you understand the noise:

await result.screenshot('albums-default', {
  threshold: 0.2,
  maxDiffPixelRatio: 0.01,
});

threshold controls how different one pixel can be before it counts. maxDiffPixelRatio controls how many pixels can differ before the screenshot fails.

Good defaults for MCP App UI:

• Use threshold: 0.2 for normal text and layout.
• Use maxDiffPixelRatio: 0.01 for stable app surfaces.
• Raise the ratio only for known noisy surfaces such as antialiased charts.
• Mask volatile regions instead of raising thresholds for the whole screenshot.
• Keep thresholds per screenshot when one state needs special handling.

You can set shared comparison rules in the sunpeak Playwright config instead of repeating them in every test:

// playwright.config.ts
import { defineConfig } from 'sunpeak/test/config';

export default defineConfig({
  hosts: ['chatgpt', 'claude'],
  visual: {
    threshold: 0.2,
    maxDiffPixelRatio: 0.01,
    snapshotPathTemplate:
      '{testDir}/__screenshots__/{projectName}/{testFilePath}/{arg}{ext}',
  },
});

These options pass through to Playwright’s toHaveScreenshot() matcher. Playwright disables CSS animations and hides carets for screenshot assertions by default, but your app can still have JavaScript timers, animated canvas content, video, network races, or font swaps. Stabilize those at the source instead of relying on the matcher.

Handle CSP, fonts, and external assets

MCP Apps often load images, fonts, maps, or API-backed media inside an iframe. The current portable resource contract uses _meta.ui.csp to tell the host which external origins the View expects. OpenAI’s current Plugins guidance starts with that MCP Apps metadata and reserves window.openai feature detection for ChatGPT-only additions.

That matters for visual testing because a missing CSP domain can look like a visual bug: empty images, fallback fonts, blank map tiles, or missing icons.

Before approving a baseline, check that:

• All image and font domains are allowed by your app resource metadata.
• Failed network requests are treated as test failures.
• Remote assets are mocked when the visual layout does not depend on the live asset.
• Your fallback UI is tested separately from your successful UI.

If a screenshot silently approves a blank image or fallback font, the baseline has encoded a bug.

Add one deliberate asset-failure state for any image, font, map, or chart that carries meaning. The successful baseline proves the asset can load; the fallback baseline proves the app stays understandable when it cannot.

Run visual regression tests in CI

Visual tests are most useful when they run on every pull request that can change UI. Keep them deterministic enough to run in CI, then upload artifacts when they fail.

# .github/workflows/test.yml
jobs:
  visual:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: 'pnpm'
      - run: pnpm install --frozen-lockfile
      - run: pnpm exec playwright install --with-deps chromium
      - run: pnpm test:visual
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: visual-regression-artifacts
          path: |
            test-results/
            tests/__screenshots__/

The artifacts matter. A failed visual test without the baseline, actual screenshot, diff, and trace leaves reviewers guessing. Store enough evidence for someone to decide whether the diff is intended.

Keep comparison and baseline generation as separate operations:

# Pull requests and main: compare only
pnpm test:visual

# A deliberate baseline update after review
npx sunpeak test --visual --update

Do not let a normal CI job update and commit snapshots automatically. A visual failure is a request for review, and auto-accepting the new image removes that review gate.

Generate baselines where CI runs

macOS and Linux render fonts differently. If CI runs on Ubuntu, generate baselines on Ubuntu. You have three practical options:

1. Generate baselines in CI, download the artifact, review it, and commit the images.
2. Run local visual tests inside the same Docker image used by CI.
3. Accept local/CI differences only for specific screenshots with narrow thresholds or masks.

The first option is easiest for most teams. The second option is better when designers and engineers need reliable local review.

Review baseline changes like code

A baseline update is a UI change. Review it with the same care as the component diff.

Ask:

• Did the intended part of the UI change?
• Did spacing, color, or typography change somewhere unrelated?
• Did text wrap differently in a host or display mode?
• Did a missing image or fallback font get approved?
• Did the screenshot include enough host context to catch the bug class we care about?
• Did the baseline update happen in the same environment CI uses?

Only commit baseline images that answer those questions cleanly.

Triage a failed screenshot in the right order

A diff image tells you where pixels changed, not why. Triage failures in this order:

1. Confirm the test used the expected host project, browser, operating system, locale, fixture, and production or development resource build.
2. Read the browser console and failed requests. Fix missing assets, CSP rejections, and bridge errors before changing a baseline.
3. Compare the semantic state. Check headings, control names, counts, and error messages so a data regression does not look like a harmless layout shift.
4. Inspect expected, actual, and diff images at the same scale. Look outside the intended component because font and spacing changes often spread.
5. Update the baseline only when the new state is correct and the test inputs did not drift.

This order separates environment noise, runtime failures, content bugs, and intentional design changes. It also gives a reviewer evidence beyond one pink diff overlay.

What a local baseline cannot prove

A local host replica can prove your own resource, data, and bridge behavior under known host inputs. It cannot prove the exact production shell, a newly released host font, account policy, model tool choice, tunnel behavior, authentication, or mobile webview version.

Use three layers:

1. Run local visual comparisons on every pull request for deterministic app states.
2. Run a small production-resource suite in the same browser and operating system as CI.
3. Before a release, connect the deployed server to each supported host and check a few prompts that cover discovery, tool selection, inline rendering, a wide mode, and one interactive action.

For ChatGPT, follow OpenAI’s current connect and test workflow. For MCP Apps generally, the official testing guide distinguishes a local reference host from a real conversational host. Local baselines should carry most of the load, while live checks cover the narrow boundary that only the host owns.

What to screenshot first

Start small. A useful visual suite usually has fewer screenshots than people expect.

For most MCP Apps, begin with:

• Default state with representative data
• Empty state
• Error state
• Dark mode
• Picture-in-picture or the tightest supported display mode
• Fullscreen or the widest supported display mode
• One long-content case

Add more only after a real bug shows that the suite has a blind spot. Visual tests are high-signal when every screenshot protects a state you would manually check before shipping.

A complete example

This test file covers the states that usually break first.

// tests/e2e/albums.visual.spec.ts
import { expect, test } from 'sunpeak/test';

test('default state', async ({ inspector }) => {
  const result = await inspector.renderTool('show-albums', {
    artist: 'Radiohead',
  });

  await expect(result.app().getByText('OK Computer')).toBeVisible();
  await result.screenshot('albums-default');
});

test('dark mode', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { theme: 'dark' },
  );

  await result.screenshot('albums-dark');
});

test('empty results', async ({ inspector }) => {
  const result = await inspector.renderTool('show-albums', {
    artist: 'Unknown Artist With No Albums',
  });

  await expect(result.app().getByText('No albums found')).toBeVisible();
  await result.screenshot('albums-empty');
});

test('error state', async ({ inspector }) => {
  const result = await inspector.renderTool('show-albums', {
    artist: 'Fixture: Server Error',
  });

  await expect(result.app().getByText('Could not load albums')).toBeVisible();
  await result.screenshot('albums-error');
});

test('picture-in-picture', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { displayMode: 'pip' },
  );

  await result.screenshot('albums-pip');
});

test('fullscreen', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { displayMode: 'fullscreen' },
  );

  await result.screenshot('albums-fullscreen');
});

When that suite runs against ChatGPT and Claude projects, it gives you host-specific baselines for the states most likely to break. That is enough coverage to catch common visual regressions without turning every CSS change into a large review job.

Current version boundary

This guide was checked on September 16, 2026 against sunpeak 0.20.84 and its shipped Playwright fixtures. The current @modelcontextprotocol/ext-apps release on npm is 2.0.0, and the MCP Apps documentation identifies 2026-01-26 as the stable protocol version. The raw SDK and sunpeak have separate release lines, so check both when a copied type or option does not match your project.

The examples rely on current sunpeak behavior: --update implies visual mode, screenshots capture the app body by default, target: 'page' is deprecated and ignored, host projects create separate baseline paths, and visual config values pass through to Playwright’s screenshot matcher.

Get started

sunpeak is an open-source MCP App framework and testing framework for MCP Apps, ChatGPT Apps, and Claude Connectors. It lets you run E2E and visual regression tests against replicated host runtimes locally and in CI, with no paid host accounts or AI credits in the default loop.

npx sunpeak new
pnpm test:visual

For an existing MCP server:

npx sunpeak test init --server http://localhost:8000/mcp
npx sunpeak test --visual

Use the testing framework for the current sunpeak testing workflow, or read the complete testing guide to place visual regression alongside unit tests, E2E tests, live host tests, and evals.