Visual Regression Testing for MCP Apps, ChatGPT Apps, and Claude Connectors (June 2026)

Your MCP App can pass unit tests, schema tests, and snapshot tests, then still look broken in the host. A gap change clips a button in picture-in-picture. A dark-mode token resolves differently in Claude. A ChatGPT display mode gives you less width than your local browser tab. The markup is valid, but the user sees a broken UI.

Visual regression testing catches that class of bug. It renders your app in a real browser, captures screenshots, and compares them to approved baselines. When something visible changes, you get a diff image instead of a vague “looks off” report.

TL;DR: Add result.screenshot() to the MCP App states that matter, run pnpm test:visual or npx sunpeak test --visual, and keep separate baselines for host, theme, display mode, and data state. Use deterministic simulation data, disable animations, upload visual artifacts in CI, and update baselines only after reviewing the diff. sunpeak runs these checks in replicated ChatGPT and Claude runtimes, so the default loop does not need paid host accounts or AI credits.

Why visual regression testing matters for MCP Apps

MCP Apps are embedded web apps. They render through resources, iframes, host bridge APIs, host CSS variables, display modes, and content security policy metadata. That makes them more sensitive to host context than a normal single-page app.

The visual surface can change because of:

• Your CSS, component library, or layout code
• Tool result data, including long text, empty states, and malformed fields
• Host theme variables in light and dark mode
• Display modes such as inline, fullscreen, and picture-in-picture
• Iframe sizing, safe areas, scroll containers, and host chrome
• Remote images, custom fonts, and CSP allowlists
• Browser and operating system rendering differences between local machines and CI

E2E tests tell you whether the app works. Visual tests tell you whether the app still looks like something a user can use.

What changed since the first version of this post

The MCP App ecosystem is more concrete now. OpenAI’s Apps SDK reference documents app metadata such as openai/widgetDescription, openai/widgetCSP, openai/widgetPrefersBorder, tool result _meta, and client-provided keys such as openai/locale. The official MCP Apps extension documentation covers host bridge behavior, resources, tool calls, and app lifecycle. Claude Connectors have also become a real distribution surface for MCP servers and interactive app experiences.

That affects visual testing in a practical way. You are not only testing a React component. You are testing a component rendered through a host contract. The baseline should include the assumptions that contract gives you: which resources load, which domains are allowed, which display modes you support, which host context values are present, and what tool result state the UI receives.

sunpeak helps because it treats those host states as test inputs. You can run local inspector tests against replicated ChatGPT and Claude runtimes, pin tool states with simulation files, and run visual regression tests in CI without driving a live AI host on every pull request.

How visual regression testing works

The workflow is simple:

1. Render an MCP resource through a tool result or simulation.
2. Wait for stable app state, including data, images, and fonts.
3. Capture a screenshot of the app iframe, the full host page, or a specific element.
4. Compare the image against a committed baseline.
5. Fail the test when the diff crosses the configured threshold.
6. Store the baseline, actual screenshot, diff image, and trace for review.

The first run creates baselines. Later runs compare against those baselines. When you make an intentional UI change, update the baseline and commit it with the code change.

Write your first visual regression test

If your project already uses the inspector fixture from sunpeak/test, visual testing is one extra line in an E2E test.

// tests/e2e/albums.visual.spec.ts
import { expect, test } from 'sunpeak/test';

test('albums resource renders the default state', async ({ inspector }) => {
  const result = await inspector.renderTool('show-albums', {
    artist: 'Radiohead',
  });

  const app = result.app();
  await expect(app.getByText('OK Computer')).toBeVisible();

  await result.screenshot('albums-default');
});

In a sunpeak app, run:

pnpm test:visual

For an existing MCP server in any language, scaffold a test project first:

npx sunpeak test init --server http://localhost:8000/mcp
npx sunpeak test --visual

Generate or refresh baselines only after you have reviewed the rendered screenshots:

npx sunpeak test --visual --update

Choose screenshot targets carefully

Most visual tests should capture the app iframe because that keeps the baseline focused on your UI. Capture the full host page when the host shell, message layout, or iframe boundary matters. Capture a single element when one component is noisy or expensive to stabilize.

// App iframe, usually the best default
await result.screenshot('albums-default');

// Full host-style page, useful for conversation shell regressions
await result.screenshot('albums-in-chat', { target: 'page' });

// Specific element, useful for focused component checks
await result.screenshot('save-button', {
  element: result.app().getByRole('button', { name: 'Save' }),
});

A good naming scheme includes the state, host, theme, and display mode when those are not already encoded by the test project:

albums-default-light-inline
albums-empty-dark-inline
albums-overflow-dark-pip
albums-error-light-fullscreen

Test the host matrix that can break

Do not screenshot every possible combination. Pick the combinations where the UI can realistically break and where a diff would change your release decision.

Hosts

Keep separate baselines for ChatGPT and Claude when you support both. The app code can be portable, but the host shell, spacing, theme values, and iframe constraints can differ.

tests/__screenshots__/
  chatgpt/
    albums.visual.spec.ts/
      albums-default.png
  claude/
    albums.visual.spec.ts/
      albums-default.png

Separate baselines are better than a single “universal” baseline because they make host-specific regressions obvious.

Themes

Take dark-mode screenshots for any surface that uses host CSS variables, semantic colors, borders, shadows, or charts.

test('albums render in dark mode', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { theme: 'dark' },
  );

  await result.screenshot('albums-dark');
});

Dark mode catches problems that text assertions miss: invisible text, low-contrast dividers, hardcoded backgrounds, and icons that disappear against the host shell.

Display modes

Display modes change viewport constraints, scroll behavior, and how much visual chrome surrounds your app. Inline mode is usually narrow. Fullscreen mode exposes wide layouts. Picture-in-picture has tight space and tends to reveal overflow.

test('albums handle picture-in-picture constraints', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { displayMode: 'pip' },
  );

  await result.screenshot('albums-pip');
});

If your app asks the host to change display mode through a bridge API, test the result after the host has accepted or rejected that request. Hosts can choose how to handle display-mode requests, so your UI should still work when the host keeps the current mode.

Build deterministic test states

Visual tests fail when the screenshot changes. That is the point, but it means volatile data has to be removed or controlled.

Use simulation files or fixtures for:

• Tool input and tool result data
• Empty states and error states
• User profile names, avatar URLs, and timestamps
• Server tool responses triggered by app actions
• Long labels, long numbers, and localized strings

Avoid live APIs in baseline tests. If you need a live API smoke test, keep it outside the visual baseline suite or mask the volatile region.

Stabilize screenshots before comparing pixels

Most flaky visual tests come from timing, not from the diff engine. Before you capture, make sure the page is ready.

test('dashboard visual state is stable', async ({ inspector }) => {
  const result = await inspector.renderTool('show-dashboard', {
    accountId: 'acct_visual_fixture',
  });

  const app = result.app();
  await expect(app.getByRole('heading', { name: 'Pipeline' })).toBeVisible();
  await expect(app.getByTestId('loading-skeleton')).toBeHidden();

  await result.screenshot('dashboard-default', {
    animations: 'disabled',
    threshold: 0.2,
    maxDiffPixelRatio: 0.01,
  });
});

Use this checklist before adding a screenshot:

• Data is fixed and comes from a simulation or fixture.
• Animations, transitions, cursors, and skeletons are gone or disabled.
• Fonts have loaded.
• Images are local, mocked, or already loaded.
• Timestamps and relative dates are frozen or hidden.
• Viewport size and display mode are explicit.
• Scroll position is controlled.
• Host, theme, and locale are explicit when they affect rendering.

Configure thresholds with intent

Pixel-perfect comparisons sound clean, but they break too often across machines. Anti-aliasing, font hinting, browser versions, and GPU differences can produce harmless pixel changes.

Start strict, then loosen only where you understand the noise:

await result.screenshot('albums-default', {
  threshold: 0.2,
  maxDiffPixelRatio: 0.01,
});

threshold controls how different one pixel can be before it counts. maxDiffPixelRatio controls how many pixels can differ before the screenshot fails.

Good defaults for MCP App UI:

• Use threshold: 0.2 for normal text and layout.
• Use maxDiffPixelRatio: 0.01 for stable app surfaces.
• Raise the ratio only for known noisy surfaces such as antialiased charts.
• Mask volatile regions instead of raising thresholds for the whole screenshot.
• Keep thresholds per screenshot when one state needs special handling.

Handle CSP, fonts, and external assets

MCP Apps often load images, fonts, maps, analytics, or API-backed media inside an iframe. In ChatGPT Apps, resource metadata such as openai/widgetCSP tells the host which domains the component expects to use. Other MCP App hosts have their own resource and security behavior.

That matters for visual testing because a missing CSP domain can look like a visual bug: empty images, fallback fonts, blank map tiles, or missing icons.

Before approving a baseline, check that:

• All image and font domains are allowed by your app resource metadata.
• Failed network requests are treated as test failures.
• Remote assets are mocked when the visual layout does not depend on the live asset.
• Your fallback UI is tested separately from your successful UI.

If a screenshot silently approves a blank image or fallback font, the baseline has encoded a bug.

Run visual regression tests in CI

Visual tests are most useful when they run on every pull request that can change UI. Keep them deterministic enough to run in CI, then upload artifacts when they fail.

# .github/workflows/test.yml
jobs:
  visual:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: pnpm/action-setup@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: 'pnpm'
      - run: pnpm install --frozen-lockfile
      - run: pnpm exec playwright install --with-deps chromium
      - run: pnpm test:visual
      - uses: actions/upload-artifact@v4
        if: failure()
        with:
          name: visual-regression-artifacts
          path: |
            test-results/
            tests/__screenshots__/

The artifacts matter. A failed visual test without the baseline, actual screenshot, diff, and trace leaves reviewers guessing. Store enough evidence for someone to decide whether the diff is intended.

Generate baselines where CI runs

macOS and Linux render fonts differently. If CI runs on Ubuntu, generate baselines on Ubuntu. You have three practical options:

1. Generate baselines in CI, download the artifact, review it, and commit the images.
2. Run local visual tests inside the same Docker image used by CI.
3. Accept local/CI differences only for specific screenshots with narrow thresholds or masks.

The first option is easiest for most teams. The second option is better when designers and engineers need reliable local review.

Review baseline changes like code

A baseline update is a UI change. Review it with the same care as the component diff.

Ask:

• Did the intended part of the UI change?
• Did spacing, color, or typography change somewhere unrelated?
• Did text wrap differently in a host or display mode?
• Did a missing image or fallback font get approved?
• Did the screenshot include enough host context to catch the bug class we care about?
• Did the baseline update happen in the same environment CI uses?

Only commit baseline images that answer those questions cleanly.

What to screenshot first

Start small. A useful visual suite usually has fewer screenshots than people expect.

For most MCP Apps, begin with:

• Default state with representative data
• Empty state
• Error state
• Dark mode
• Picture-in-picture or the tightest supported display mode
• Fullscreen or the widest supported display mode
• One long-content case

Add more only after a real bug shows that the suite has a blind spot. Visual tests are high-signal when every screenshot protects a state you would manually check before shipping.

A complete example

This test file covers the states that usually break first.

// tests/e2e/albums.visual.spec.ts
import { expect, test } from 'sunpeak/test';

test('default state', async ({ inspector }) => {
  const result = await inspector.renderTool('show-albums', {
    artist: 'Radiohead',
  });

  await expect(result.app().getByText('OK Computer')).toBeVisible();
  await result.screenshot('albums-default');
});

test('dark mode', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { theme: 'dark' },
  );

  await result.screenshot('albums-dark');
});

test('empty results', async ({ inspector }) => {
  const result = await inspector.renderTool('show-albums', {
    artist: 'Unknown Artist With No Albums',
  });

  await expect(result.app().getByText('No albums found')).toBeVisible();
  await result.screenshot('albums-empty');
});

test('error state', async ({ inspector }) => {
  const result = await inspector.renderTool('show-albums', {
    artist: 'Fixture: Server Error',
  });

  await expect(result.app().getByText('Could not load albums')).toBeVisible();
  await result.screenshot('albums-error');
});

test('picture-in-picture', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { displayMode: 'pip' },
  );

  await result.screenshot('albums-pip');
});

test('fullscreen', async ({ inspector }) => {
  const result = await inspector.renderTool(
    'show-albums',
    { artist: 'Radiohead' },
    { displayMode: 'fullscreen' },
  );

  await result.screenshot('albums-fullscreen');
});

When that suite runs against ChatGPT and Claude projects, it gives you host-specific baselines for the states most likely to break. That is enough coverage to catch common visual regressions without turning every CSS change into a large review job.

Get started

sunpeak is an open-source MCP App framework and testing framework for MCP Apps, ChatGPT Apps, and Claude Connectors. It lets you run E2E and visual regression tests against replicated host runtimes locally and in CI, with no paid host accounts or AI credits in the default loop.

npx sunpeak new
pnpm test:visual

For an existing MCP server:

npx sunpeak test init --server http://localhost:8000/mcp
npx sunpeak test --visual

Use the testing framework for the current sunpeak testing workflow, or read the complete testing guide to place visual regression alongside unit tests, E2E tests, live host tests, and evals.