> ## Documentation Index
> Fetch the complete documentation index at: https://sunpeak.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# MCP Testing Framework

> Test any MCP server with E2E, live host, eval, and unit testing. Works standalone or as part of a sunpeak framework project.

The sunpeak testing framework tests any MCP server, whether or not you use the sunpeak app framework. It provides four levels of automated testing that cover everything from fast unit checks to live host validation and multi-model eval runs.

## Quickstart

No sunpeak project required. Scaffold tests for any running MCP server:

```bash theme={null}
npx sunpeak test init --server http://localhost:8000/mcp
npx sunpeak test
```

This generates test infrastructure (Playwright configs, E2E specs, visual regression, eval boilerplate) and runs the scaffolded smoke test. See [Getting Started](/testing/getting-started) for the full setup walkthrough, including language-specific tips for Python, Go, and Rust servers.

For sunpeak framework projects, the dev server starts automatically and no server URL is needed.

## Testing Levels

### 1. E2E Tests

Playwright specs that call your MCP tools and render them in simulated ChatGPT and Claude runtimes. The `mcp` fixture from `sunpeak/test` covers protocol calls (`callTool`, `listTools`, `listResources`, `readResource`), while the `inspector` fixture handles rendering: inspector navigation, double-iframe traversal, and host switching via `renderTool`. [Simulations](/testing/simulations) (JSON fixtures) define reproducible tool states so you can test every combination of host, theme, display mode, and device without deploying or burning API credits.

Visual regression is built in. Pass `--visual` to compare screenshots against baselines, or `--visual --update` to regenerate them.

<Card horizontal title="E2E Testing" icon="flask" href="/testing/e2e">
  Write Playwright tests against simulated ChatGPT and Claude runtimes.
</Card>

<Card horizontal title="Visual Regression" icon="image" href="/testing/visual">
  Screenshot comparison across themes, display modes, and hosts.
</Card>

### 2. Live Tests

Playwright specs that run against real ChatGPT (and future hosts). They open a browser, send messages that trigger tool calls against your MCP server, and verify the rendered app. This catches issues that inspector tests cannot: real MCP connection behavior, actual LLM tool invocation, host-specific iframe rendering, and production resource loading.

<Card horizontal title="Live Testing" icon="globe" href="/testing/live">
  Validate your MCP Apps inside real AI chat hosts.
</Card>

### 3. Evals

Multi-model tool calling tests. Evals connect to your MCP server via the MCP protocol, discover its tools, and send prompts to multiple LLM models (GPT-4o, Claude, Gemini, etc.). Each eval case runs N times per model and reports statistical pass/fail counts, so you can measure whether your tool descriptions work reliably across models.

<Card horizontal title="Evals" icon="chart-bar" href="/testing/evals">
  Test tool calling reliability across GPT-4o, Claude, Gemini, and more.
</Card>

## CLI Commands

| Command                          | What it runs                     | Runtime                |
| -------------------------------- | -------------------------------- | ---------------------- |
| `sunpeak test`                   | Unit (if configured) + E2E tests | Vitest + Playwright    |
| `sunpeak test --e2e`             | E2E tests only                   | Playwright + inspector |
| `sunpeak test --visual`          | E2E with visual regression       | Playwright + inspector |
| `sunpeak test --visual --update` | Update visual baselines          | Playwright + inspector |
| `sunpeak test --live`            | Live tests against real hosts    | Playwright + real host |
| `sunpeak test --eval`            | Evals against multiple models    | Vitest + Vercel AI SDK |
| `sunpeak test --unit`            | Unit tests (app framework only)  | Vitest + happy-dom     |

Flags are additive: `--e2e --live --eval` runs all three.

<Note>
  `--eval` and `--live` are not included in the default `sunpeak test` run because they require API keys and cost money. You must opt in explicitly.
</Note>

## Scaffolding

For existing MCP servers (not built with sunpeak), run `npx sunpeak test init` to generate all the test infrastructure:

```bash theme={null}
npx sunpeak test init
```

For JS/TS projects, this creates files at the project root:

* `tests/e2e/` with smoke and visual regression test specs
* `tests/evals/` with eval config, `.env.example`, and example eval specs
* `tests/live/` with live test config and example specs

For non-JS projects (Python, Go, Rust, etc.), everything goes into a self-contained `tests/sunpeak/` directory with its own `package.json`.

For sunpeak framework projects, `sunpeak new` scaffolds all of this automatically.

## Learn More

<CardGroup cols={2}>
  <Card title="Inspector" icon="desktop" href="/testing/inspector">
    The multi-host inspector that powers E2E tests.
  </Card>

  <Card title="Simulations" icon="folder" href="/testing/simulations">
    JSON fixtures for reproducible tool states.
  </Card>

  <Card title="E2E Testing" icon="flask" href="/testing/e2e">
    Playwright specs against simulated hosts.
  </Card>

  <Card title="Visual Regression" icon="image" href="/testing/visual">
    Screenshot baselines and comparison.
  </Card>

  <Card title="Live Testing" icon="globe" href="/testing/live">
    Tests against real ChatGPT and Claude.
  </Card>

  <Card title="Evals" icon="chart-bar" href="/testing/evals">
    Multi-model tool calling reliability.
  </Card>
</CardGroup>