MCP App CI/CD: Run Your Tests in GitHub Actions (September 2026)

[Updated 2026-09-05] A useful MCP App pipeline answers three different questions: does the server follow the MCP contract, does the UI work inside each host runtime, and does the deployed app still work in a real host? Those questions need different test jobs because they have different trust, cost, and failure boundaries.

TL;DR: Run static checks, unit tests, inspector E2E tests, and a production build on every pull request. Use sunpeak’s replicated ChatGPT and Claude runtimes for the broad test matrix, so the default gate stays deterministic and needs no paid host accounts. Put visual checks in the pull request path when layout matters. Run live ChatGPT smoke tests and model evals only in trusted jobs with tightly scoped credentials.

What an MCP App Pipeline Must Prove

MCP Apps combine a server contract, a sandboxed web UI, and a host bridge. A green component test alone does not prove that a host can discover the tool, read its UI resource, deliver tool data, or receive app messages.

The MCP Apps testing guide recommends testing with a reference host and then checking the app in a compatible conversational host. That split maps cleanly to CI:

Layer	What it catches	Where it runs
Static checks	Type errors, lint failures, invalid imports	Every pull request
Unit tests	Hooks, components, server helpers, pure business logic	Every pull request
MCP contract tests	Tool schemas, resource metadata, results, errors	Every pull request
Inspector E2E	Host bridge, iframe UI, themes, display modes, app actions	Every pull request
Visual regression	Layout drift, clipping, overflow, missing states	UI pull requests
Live host smoke	Auth, remote connectivity, rollout, real host behavior	Trusted branch or manual run
Model evals	Tool selection and argument quality	Schema changes, schedule, or manual run

The first five layers can be deterministic. Live checks and evals depend on external systems, so they should report a distinct failure instead of turning every pull request into a remote integration test.

Start With a Secure Pull Request Gate

This workflow is a practical default for a current sunpeak project. The action references are pinned to the full commit behind their September 2026 releases because GitHub says a full-length commit SHA is the only immutable action reference. The comments preserve the readable release number for updates.

name: MCP App CI

on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read

concurrency:
  group: mcp-app-ci-${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    name: Unit, E2E, and build
    runs-on: ubuntu-latest
    timeout-minutes: 20

    steps:
      - name: Check out repository
        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

      - name: Set up pnpm
        uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
        with:
          version: 11

      - name: Set up Node.js
        uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
        with:
          node-version: 24
          cache: pnpm
          cache-dependency-path: pnpm-lock.yaml

      - name: Install dependencies
        run: pnpm install --frozen-lockfile

      - name: Install Chromium
        run: pnpm exec playwright install --with-deps chromium

      - name: Run static checks
        run: pnpm typecheck && pnpm lint

      - name: Run unit and inspector E2E tests
        run: pnpm test

      - name: Build production assets
        run: pnpm build

      - name: Upload browser evidence
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        with:
          name: browser-evidence-${{ github.run_attempt }}
          path: |
            playwright-report/
            test-results/
          if-no-files-found: ignore
          retention-days: 7

Adjust the script names to match your repository. A new sunpeak project provides test, test:unit, test:e2e, test:visual, test:live, and test:eval; your own typecheck and lint scripts may use different names.

The important controls are easy to miss:

• permissions: contents: read gives the workflow only the repository access it needs.
• concurrency cancels stale runs after a new commit lands on the same branch.
• --frozen-lockfile stops CI from silently changing dependency resolution.
• The production build catches resource bundling errors that dev mode can hide.
• if: ${{ !cancelled() }} keeps traces and reports after failures and still skips artifact work after cancellation, which matches Playwright’s current CI example.

Use the Node and pnpm versions your production environment supports. Pin the package manager in package.json, then test any additional Node version in a small matrix if your published server promises more than one runtime.

Test the MCP Contract Before the Browser

Browser failures are slower to diagnose than direct protocol failures. Test these server responses without rendering the UI:

• tools/list exposes the expected name, description, input schema, annotations, and UI resource link.
• resources/read returns the expected URI, MIME type, and bundled HTML.
• tools/call returns valid content, structuredContent, _meta, and isError values for success and failure cases.
• Schemas reject malformed input and keep destructive or read-only hints accurate.
• Private data stays in _meta when the model should not see it.

Then use the inspector for behavior that requires a host runtime. This order turns a vague blank-iframe failure into a specific contract assertion whenever the server response is the cause.

Run One E2E Spec Across Host Projects

sunpeak’s Playwright helper starts the app or inspector, waits for its health check, and creates ChatGPT and Claude projects by default:

// playwright.config.ts
import { defineConfig } from 'sunpeak/test/config';

export default defineConfig({
  hosts: ['chatgpt', 'claude'],
  workers: process.env.CI ? 1 : undefined,
});

Playwright recommends one worker in CI for stability and reproducibility. Start there, record the duration, and shard the suite only when runtime justifies the added coordination.

Host selection belongs to the Playwright project. renderTool() selects the state to render, such as theme and display mode:

import { expect, test } from 'sunpeak/test';

test('dashboard renders a weekly total', async ({ inspector }) => {
  const result = await inspector.renderTool('get-dashboard', undefined, {
    displayMode: 'inline',
    theme: 'dark',
    prodResources: true,
  });

  const app = result.app();
  await expect(app.getByText('4,218')).toBeVisible();
});

Do not pass a host option to renderTool(). That pattern looks plausible but does not select a sunpeak host. The same test runs once per configured Playwright project, and the inspector.host fixture tells you which project is active when a host-specific assertion is necessary.

Use prodResources: true in at least one gate. It loads the production resource bundle instead of relying only on the development path, which catches stale assets, missing chunks, and build-only CSP mistakes.

Turn Model-Driven States Into Fixtures

An MCP App can receive empty results, partial data, structured errors, authorization prompts, and large datasets. Waiting for a model or backend to produce each state makes the suite slow and unreliable. Store each state as a simulation instead:

{
  "tool": "get_dashboard",
  "userMessage": "Show me this week's analytics",
  "toolInput": { "timeRange": "7d" },
  "toolResult": {
    "structuredContent": {
      "visits": 4218,
      "conversions": 83,
      "bounceRate": 0.41
    }
  }
}

A useful simulation set covers real behavior, not just lines of code:

• Normal data with realistic field lengths.
• No records and no search results.
• Backend and tool errors.
• Missing optional fields and partial structured content.
• Long labels, large values, and dense collections.
• Authenticated, unauthenticated, and permission-denied states.
• Every display mode that changes the layout.

These fixtures make failures reproducible locally and give reviewers a concrete inventory of supported UI states. See the MCP App testing strategy for deciding which combinations belong in unit, integration, E2E, and live tests.

Add Visual Regression Deliberately

Run pnpm test:visual when spacing, clipping, chart geometry, or responsive behavior can break without changing text assertions. Store approved baselines in the repository and review image diffs as code review evidence.

  visual:
    name: Visual regression
    runs-on: ubuntu-latest
    timeout-minutes: 20

    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
        with:
          version: 11
      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
        with:
          node-version: 24
          cache: pnpm
      - run: pnpm install --frozen-lockfile
      - run: pnpm exec playwright install --with-deps chromium
      - run: pnpm test:visual
      - name: Upload visual evidence
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        with:
          name: visual-evidence-${{ github.run_attempt }}
          path: |
            playwright-report/
            test-results/
          if-no-files-found: ignore
          retention-days: 7

Avoid updating screenshots in CI. Generate a candidate baseline locally, inspect the before-and-after images, and commit an intentional change with the code that caused it. The visual regression guide covers stable data, fonts, animations, viewports, and host-state matrices.

Cache Dependencies, Not Assumptions

actions/setup-node can cache pnpm’s store from the lockfile, which speeds downloads without skipping the install or lockfile check. For a monorepo, set cache-dependency-path to the workspace lockfile that governs the job.

Do not cache ~/.cache/ms-playwright by default. Playwright says browser binary caching is generally not recommended because restoring the cache can take about as long as downloading it, while Linux system packages still need installation. Install only the browsers the suite uses:

pnpm exec playwright install --with-deps chromium

Measure first if browser installation becomes a material part of the job. A cache that saves no wall-clock time still adds invalidation rules and another failure path.

Keep Secrets Out of Untrusted Code

Inspector tests need no host or model credentials, so the pull request workflow should remain secret-free. This matters for public repositories and any organization that accepts fork pull requests.

GitHub withholds repository secrets from fork pull requests and limits GITHUB_TOKEN to read-only access. Do not work around that with pull_request_target while checking out and running the pull request’s code. GitHub warns that this gives untrusted code access to a privileged workflow context.

Use a separate trusted workflow for live tests, evals, and deployment. Give each job only the permissions it needs. Prefer OpenID Connect for cloud deployment because it exchanges the workflow identity for a short-lived token instead of storing a long-lived cloud credential in GitHub.

Make Live Tests Narrow and Explicit

As of September 2026, sunpeak’s default live adapter targets real ChatGPT. Use it to check the integration boundary that the local replica cannot own: account authorization, remote server reachability, real host rollout state, and final iframe rendering.

  live:
    name: Live ChatGPT smoke
    if: github.event_name == 'workflow_dispatch'
    runs-on: [self-hosted, linux, x64]
    environment: live-host-testing
    permissions:
      contents: read

    steps:
      - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
      - uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
        with:
          version: 11
      - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7.0.0
        with:
          node-version: 24
          cache: pnpm
      - run: pnpm install --frozen-lockfile
      - run: pnpm exec playwright install --with-deps chromium
      - name: Run live smoke tests
        run: pnpm test:live

Use a trusted desktop runner because the current ChatGPT live flow opens a real browser and may require a person to log in again when the saved session expires. Do not commit the generated .auth directory or upload it as a normal workflow artifact. If your runner restores an encrypted Playwright storage-state file, point SUNPEAK_STORAGE_STATE at that file, but expect some anti-bot cookies to require a fresh interactive session.

Keep this suite short: connect the production MCP endpoint, invoke one representative UI tool, verify the resource renders, and check one action back to the server. The inspector should own the broad host, theme, display mode, data, and error matrix.

For final ChatGPT checks, follow OpenAI’s current connect and test instructions. Deployment UI and distribution can change faster than the MCP contract, which is another reason to keep live tests isolated from the deterministic gate.

Run Evals at the Model Boundary

Unit and E2E tests can prove that a tool works after invocation. They cannot prove that a model chooses the tool for the right request or supplies useful arguments.

Run pnpm test:eval after changes to:

• Tool names and descriptions.
• Input schemas and required fields.
• Tool annotations and auth hints.
• Model-visible result content.
• The set of tools exposed together.

Pin model identifiers, keep a fixed prompt corpus, set explicit pass thresholds, and record cost and latency with the result. Put exact schema and policy checks in deterministic tests; use evals for judgment calls that need a model. This keeps model variance from masking a protocol regression.

Build Once, Then Promote the Same Artifact

The pull request gate proves source behavior. A release workflow should prove that the exact production bundle passed before deployment:

1. Build the server and UI resources once from the release commit.
2. Test the production bundle with prodResources: true.
3. Upload a uniquely named, immutable artifact and record its commit SHA or digest.
4. Require the test jobs before the deploy job.
5. Use a protected production environment and a deployment concurrency group.
6. Deploy the tested artifact instead of rebuilding with a different dependency state.
7. Run a small post-deploy MCP contract check, then a live host smoke test when warranted.

GitHub environments can restrict deployment branches, require approval, and hold environment secrets until protection rules pass. Set deployment concurrency separately from pull request concurrency so two releases cannot race each other.

Keep the previous artifact or release reference available for rollback. A rollback plan based on the exact last-known-good build is much faster than trying to recreate it after an incident.

Preserve Release Evidence for SOC 2 Work

If your MCP App is in scope for SOC 2 work, keep a release record that connects the pull request, reviewer approval, commit SHA, CI result, and deployed artifact. Include the host projects and test configuration so someone reviewing the record can see which behavior the tests covered. The seven-day artifact retention in the examples above is a debugging default; choose evidence retention separately with the people responsible for your program.

FileGRC is a Git-native GRC workspace that keeps structured records in JSON, long-form work in Markdown, and change history in Git. Its guide to SOC 2 in Git explains how to organize program records and fixed evidence collected from source systems. Your team still collects the CI evidence, and the CPA firm decides whether it is sufficient for the audit.

Test an Existing MCP Server

sunpeak’s testing framework is server-agnostic. You can add the same CI layers to an existing Python, Go, Rust, or TypeScript server without moving the server into the sunpeak application framework.

For an HTTP server:

npx sunpeak test init --server http://localhost:8000/mcp

For a stdio server:

npx sunpeak test init --server "python server.py"

Commit the generated harness and start any database, queue, or backend dependency before Playwright. Use readiness checks rather than fixed sleeps. The generated config can start a command itself, or it can connect to an HTTP server started by the workflow.

See the sunpeak testing framework for the current command set and the E2E testing guide for fixture and server configuration patterns.

Use This Release Gate

A practical MCP App release policy is:

1. Every pull request runs static checks, unit tests, MCP contract tests, inspector E2E, and a production build.
2. UI changes run visual tests against deterministic simulations.
3. No pull request job needs a host session, provider key, or deployment credential.
4. Trusted tool-contract changes run pinned-model evals with explicit thresholds.
5. A protected deploy job promotes the already-tested artifact.
6. Post-deploy checks verify the MCP endpoint, with a narrow live ChatGPT smoke test when the release affects host integration.

This structure gives each failure a clear owner. Protocol failures point to the server contract, inspector failures point to app or host-runtime behavior, visual diffs point to the rendered UI, live failures point to remote integration, and eval failures point to model-facing tool design. That makes the pipeline useful during review and when a production release goes wrong.

Start with the sunpeak testing documentation, or run npx sunpeak test init --server URL to add the inspector and test harness to an existing MCP server.