协作头脑风暴
brainstorming
在实现前澄清需求、探索设计选择,并形成清晰的开发方向。
+ `allowJs`) — un-ignoring a file does nothing if no transform matches it |\n| Coverage below 95% | Untested error paths or conditional branches | Run `frontmcp test --coverage` and inspect uncovered lines in the report |\n| E2E test timeout | Server startup too slow or port conflict | Increase Jest timeout; use random port allocation |\n| DI resolution fails in tests | Provider not registered in test scope | Register mock providers before creating the test context |\n| Istanbul shows 0% on async methods | TypeScript source-map mismatch with Istanbul | Known issue with some TS compilation settings; verify coverage with actual test output |\n| Specs see no `.env` values | An older CLI did not load `.env` for `frontmcp test`, only for `dev` | Upgrade the CLI — `frontmcp test` now loads `.env` / `.env.local` with the same precedence as `dev` (real environment wins, so CI secrets still override). Pass `--no-env` for a hermetic run |\n| `Invalid first argument, true` at collection time | `test.skip(condition, reason)` reached Jest's `skip(name, fn)` | Upgrade the CLI — the Playwright signature is supported: `test.skip(!hasCredentials, 'credentials not set')` skips every test registered after it in the enclosing block |\n| `test.use()` in one `describe` leaks into another, or two files fight over a port | Older `@frontmcp/testing` kept one global config and per-process ports | Upgrade — `test.use()` is scoped per `describe` and ports are locked across Jest workers; use `port: 0` |\n| Every spec fails with `HTTP 404` after setting `http.entryPath` | The test client always connected to the server root | Upgrade the CLI — the client now follows the `entryPaths` a 404 reports, and `test.use({ entryPath: '/mcp' })` sets it explicitly |\n| `test.beforeEach(async ({ mcp }) => …)` times out waiting for `done` | Older `@frontmcp/testing` handed the hook to Jest, which read the parameter as `done` | Upgrade — hooks that take a parameter now receive the test's fixtures (same `mcp` as the test); `beforeAll` / `afterAll` never get fixtures |\n| `test.use({ transport: 'sse' })` throws \"SSE transport not yet implemented\" | Older `@frontmcp/testing` had no legacy SSE client | Upgrade — `'sse'` is the legacy HTTP+SSE transport; the server must enable it with `transport: { protocol: { legacy: true } }` |\n| `Module @swc/jest in the transform option was not found` from `frontmcp test` | Older `@frontmcp/testing` did not install the transformer the injected config uses | Upgrade — `@frontmcp/testing` depends on `@swc/jest` / `@swc/core`, and `frontmcp test` resolves the transformer through it |\n| `ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG` from a server started inside a spec | Older SDKs loaded the optional `vectoriadb` with `import()` at boot | Upgrade the SDK — `vectoriadb` is a regular dependency, imported without `import()`; no `--experimental-vm-modules` needed |\n\n## Examples\n\nEach reference has matching examples under [`examples/\u003creference>/`](./examples/):\n\n### `setup-testing`\n\n| Example | Level | Description |\n| ---------------------------------------------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- |\n| [`fixture-based-e2e-test`](./examples/setup-testing/fixture-based-e2e-test.md) | Advanced | Write E2E tests using the fixture API from `@frontmcp/testing` that manages server lifecycle automatically and uses MCP-specific custom matchers. |\n| [`jest-config-with-coverage`](./examples/setup-testing/jest-config-with-coverage.md) | Basic | Set up a Jest configuration file that enforces 95%+ coverage across all metrics for a FrontMCP library. |\n| [`unit-test-tool-resource-prompt`](./examples/setup-testing/unit-test-tool-resource-prompt.md) | Intermediate | Write unit tests for the three core MCP primitives, verifying that outputs match the expected MCP response shapes. |\n\n### `test-auth`\n\n| Example | Level | Description |\n| -------------------------------------------------------------------------- | ------------ | --------------------------------------------------------------------------------------------------------- |\n| [`oauth-flow-test`](./examples/test-auth/oauth-flow-test.md) | Advanced | Use `MockOAuthServer` to simulate an OAuth identity provider and test the authorization code flow. |\n| [`role-based-access-test`](./examples/test-auth/role-based-access-test.md) | Intermediate | Verify that tools enforce role-based access by testing admin and user tokens against protected endpoints. |\n| [`token-factory-test`](./examples/test-auth/token-factory-test.md) | Basic | Use `TestTokenFactory` to create tokens and verify authenticated and unauthenticated requests. |\n\n### `test-browser-build`\n\n| Example | Level | Description |\n| ----------------------------------------------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------ |\n| [`browser-bundle-validation`](./examples/test-browser-build/browser-bundle-validation.md) | Basic | Verify that the browser build produces a valid bundle without Node.js-only module references. |\n| [`playwright-browser-test`](./examples/test-browser-build/playwright-browser-test.md) | Advanced | Use Playwright to test a browser-based MCP client that loads and calls tools from an MCP server. |\n\n### `test-cli-binary`\n\n| Example | Level | Description |\n| ------------------------------------------------------------------------------ | ------------ | ------------------------------------------------------------------------------------------------------------------ |\n| [`binary-startup-test`](./examples/test-cli-binary/binary-startup-test.md) | Basic | Verify that a compiled CLI binary starts correctly and responds to health checks. |\n| [`js-bundle-import-test`](./examples/test-cli-binary/js-bundle-import-test.md) | Intermediate | Verify that the compiled JS bundle can be imported and exports the expected modules after a `frontmcp build` step. |\n\n### `test-direct-client`\n\n| Example | Level | Description |\n| ----------------------------------------------------------------------------------------- | ------------ | ----------------------------------------------------------------------------------------------------------------------------- |\n| [`basic-create-test`](./examples/test-direct-client/basic-create-test.md) | Basic | Test tools in-memory without any HTTP overhead using the `create()` function from `@frontmcp/sdk`. |\n| [`openai-claude-format-test`](./examples/test-direct-client/openai-claude-format-test.md) | Intermediate | Verify that tools are returned in the correct format for OpenAI and Claude clients using `connectOpenAI` and `connectClaude`. |\n\n### `test-e2e-handler`\n\n| Example | Level | Description |\n| --------------------------------------------------------------------------------------------- | ------------ | -------------------------------------------------------------------------------------------------------------------- |\n| [`basic-e2e-test`](./examples/test-e2e-handler/basic-e2e-test.md) | Basic | Set up a basic E2E test that starts a server, connects a client, and verifies tools are listed. |\n| [`manual-client-with-transport`](./examples/test-e2e-handler/manual-client-with-transport.md) | Advanced | Use `McpTestClient.create()` with explicit transport settings for fine-grained control over E2E tests. |\n| [`tool-call-and-error-e2e`](./examples/test-e2e-handler/tool-call-and-error-e2e.md) | Intermediate | Test successful tool calls and verify that invalid inputs produce proper error responses over the full MCP protocol. |\n\n### `test-tool-unit`\n\n| Example | Level | Description |\n| ----------------------------------------------------------------------------------- | ------------ | -------------------------------------------------------------------------------------------------------- |\n| [`basic-tool-test`](./examples/test-tool-unit/basic-tool-test.md) | Basic | Test a simple tool's `execute()` method with mock context and verify the output. |\n| [`schema-validation-test`](./examples/test-tool-unit/schema-validation-test.md) | Intermediate | Validate that a tool's Zod input schema rejects invalid data before `execute()` is called. |\n| [`tool-error-handling-test`](./examples/test-tool-unit/tool-error-handling-test.md) | Advanced | Test that a tool throws the correct MCP error classes with proper error codes and JSON-RPC error shapes. |\n\n## Accessing This Skill\n\nSkills are distributed as plain SKILL.md files plus a sibling `references/`\nand `examples/` tree, so consumers can pick whichever access mode fits:\n\n| Mode | How it works |\n| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| **Filesystem** | Read `libs/skills/catalog/frontmcp-testing/` directly from a clone of the catalog repo, or from a published `@frontmcp/skills` install. SKILL.md is the entry point. |\n| **`frontmcp` CLI** | `frontmcp skills list`, `frontmcp skills read frontmcp-testing`, `frontmcp skills read frontmcp-testing:references/\u003cfile>.md`, `frontmcp skills install frontmcp-testing` — no server required. |\n| **MCP `skill://`** | When a developer mounts this skill into their own FrontMCP server (`@FrontMcp({ skills: [...] })`), the SDK exposes it via SEP-2640 resources: `skill://frontmcp-testing/SKILL.md`, `skill://frontmcp-testing/references/{file}.md`, etc. The server’s `skill://index.json` returns the SEP-2640 discovery document for everything mounted on it. |\n\nThe catalog itself is **not** an MCP server. The `skill://` URIs only resolve\nwhen a server has been configured to host this skill.\n\n## Reference\n\n- [Testing Documentation](https://frontmcp.dev/reference/testing)\n- Related skills: `setup-testing`, `create-tool`, `create-resource`, `create-prompt`, `configure-auth`\n","licenseDocuments":[]}}
frontmcp-testing【开发与工程】Use for anything about testing FrontMCP servers: writing or running unit, integration, and E2E tests and reaching the 95%+ coverage bar. Covers Jest setup and coverage gating; unit-testing a ToolContext execute() with mock context, inputs, and Zod schema validation; testing resources and prompts; in-memory testing via create() and connectOpenAI / con…
独立 GitHub 快照 · 自动分类 · 以原作者文档为准
Entry point for testing FrontMCP applications. This skill helps you navigate testing strategies across component types and find the right patterns for unit, integration, and E2E tests.
setup-testing)frontmcp-development)frontmcp-deployment)Decision: Use this skill for testing strategy and routing. Open the
setup-testingreference underreferences/for hands-on Jest configuration and test writing.
frontmcp-development).setup-testing before opening any other testing skill.This is a router skill. Follow this order to pick a testing approach, then move to the target reference under references/.
references/test-tool-unit.md, references/test-e2e-handler.md, references/test-auth.md) and follow its Steps section.references/setup-testing.md).| Scenario | Reference / Section | Description |
|---|---|---|
| Set up Jest, coverage, and test harness | setup-testing | Full Jest config, test utilities, and coverage thresholds |
| Write unit tests for a tool | test-tool-unit | Mock DI, validate input/output, test error paths |
| Write unit tests for a resource | setup-testing (Unit Testing) | Test URI resolution, template params, read results |
| Write unit tests for a prompt | setup-testing (Unit Testing) | Test argument handling, message generation |
| Write E2E protocol-level tests | setup-testing (E2E Testing) | Real MCP client/server, full protocol flow |
| Test authenticated endpoints | test-auth | E2E with OAuth tokens, session validation, role-based access |
| Test deployment builds | setup-testing + deploy-to-* | Smoke tests against built output |
| Test browser builds | test-browser-build | Smoke-test a frontmcp build --target browser bundle (import the bundle, optional Playwright suite) |
| Test CLI binary builds | test-cli-binary | Spawn-and-curl smoke tests for frontmcp build --target cli artifacts |
| Test with the direct API client | test-direct-client | In-memory testing via create(), connectOpenAI(), connectClaude() (no HTTP) |
| Write E2E test handler patterns | test-e2e-handler | Manual McpTestClient + TestServer E2E patterns (alternative to fixture API) |
| Unit test individual tools | test-tool-unit | Unit testing individual ToolContext subclasses with a mock context |
| Component | Unit Test Focus | E2E Test Focus | Key Assertions |
|---|---|---|---|
| Tool | Input validation, execute logic, error paths, DI mocking | tools/call via MCP client | Output matches schema, errors return MCP codes |
| Resource | URI resolution, read content, template param handling | resources/read via MCP client | Content type correct, URI patterns resolve |
| Prompt | Argument validation, message generation, multi-turn | prompts/get via MCP client | Messages match expected structure |
| Agent | LLM config, tool selection, handoff logic | Agent loop via MCP client | Tools called in order, result synthesized |
| Provider | Lifecycle hooks, factory output, singleton behavior | Indirectly via tool/resource tests | Instance reuse, cleanup on scope disposal |
| Job | Progress tracking, retry logic, attempt counting | Job execution via test harness | Progress events emitted, retries respected |
| Workflow | Step dependencies, conditions, input mapping functions | Workflow run via test harness | Steps execute in order, conditions evaluated, continueOnError respected |
| Skill | Instruction loading (inline/file/url), tool validation | Skill discovery via MCP/HTTP | Instructions resolve, tool refs validated per toolValidation mode |
| Plugin | Context extensions, provider registration, hook firing | Indirectly via tool tests | Extensions available on this, hooks fire at correct stages |
| Pattern | Rule |
|---|---|
| File naming | Always .spec.ts (not .test.ts); E2E uses .e2e.spec.ts |
| File organization | Split E2E tests by app/feature: e2e/calc.e2e.spec.ts, e2e/ecommerce.e2e.spec.ts. Never put all tests in a single server.e2e.spec.ts |
| Environment | frontmcp test loads .env / .env.local into the Jest child, same precedence as dev (real environment wins over the files, config env.shared + env.test underneath). --no-env skips it |
| Test runner | Standalone projects: use frontmcp test (auto-generates Jest/SWC config; discovers src/**/*.spec.ts(x), __tests__/**/*.spec.ts(x), and e2e/**/*.e2e.spec.ts(x); transforms both .ts and .tsx with the automatic JSX runtime; transpiles ESM-only deps such as jose, @noble/hashes and @noble/ciphers (CodeCall) under npm, yarn AND pnpm's node_modules/.pnpm/ store — add your own via test.esmPackages in frontmcp.config.ts; does not force NODE_OPTIONS=--experimental-vm-modules (it would bypass the ESM transforms; scaffolded projects use Jest 30); delegates to a user-provided jest.config.{ts,js,mjs,cjs,json} if present, which drops the injected config unless it starts from preset: '@frontmcp/testing', which carries the same transform, discovery, timeout and ESM transforms). Nx monorepos: use nx test <lib> (resolves the project's jest.config.ts). Never invoke jest --config ... directly |
| Coverage threshold | 95%+ across statements, branches, functions, lines |
| Test descriptions | Plain English, no prefixes like "PT-001"; describe behavior not implementation |
| Mocking | Mock providers via DI token replacement, never mock the framework |
| httpMock scope | httpMock intercepts HTTP in the test process only, NOT in the MCP server subprocess. Do not use httpMock to intercept server-to-API calls — those happen in the child process. Use httpMock for verifying client-to-server request shapes or mocking external APIs called from the test itself |
| Error testing | Assert instanceof specific error class AND MCP error code |
| Async | Always await async operations; use expect(...).rejects.toThrow() for async errors |
| Pattern | Correct | Incorrect | Why |
|---|---|---|---|
| Test file location | fetch-weather.tool.spec.ts next to source | __tests__/fetch-weather.test.ts | Co-location with .spec.ts extension matches FrontMCP conventions |
| DI mocking | Replace token with mock via scope.register(TOKEN, mockImpl) | jest.mock('../provider') module mock | DI mocking is cleaner, type-safe, and tests the real integration path |
| Error assertions | expect(err).toBeInstanceOf(ResourceNotFoundError) | expect(err.message).toContain('not found') | Class checks are stable; message strings are fragile |
| E2E transport | Use @frontmcp/testing MCP client with real server | HTTP requests with fetch | The test client handles protocol details (session, framing) |
| Coverage gaps | Investigate uncovered branches, add targeted tests | Add istanbul ignore comments | Coverage gaps often hide real bugs; ignoring them defeats the purpose |
@frontmcp/testing preset.spec.ts extension throughout| Problem | Cause | Solution |
|---|---|---|
| Jest not finding test files | Wrong file extension (.test.ts instead of .spec.ts) | Rename to .spec.ts; check testMatch in jest.config |
SyntaxError: Unexpected token 'export' | An ESM-only dependency is being ignored instead of transpiled | Add it to test.esmPackages in frontmcp.config.ts. With a hand-written jest.config.ts, start from preset: '@frontmcp/testing' or use the pnpm-safe transformIgnorePatterns in setup-testing AND make sure transform matches .js (^.+\.[tj]sx?$ + allowJs) — un-ignoring a file does nothing if no transform matches it |
| Coverage below 95% | Untested error paths or conditional branches | Run frontmcp test --coverage and inspect uncovered lines in the report |
| E2E test timeout | Server startup too slow or port conflict | Increase Jest timeout; use random port allocation |
| DI resolution fails in tests | Provider not registered in test scope | Register mock providers before creating the test context |
| Istanbul shows 0% on async methods | TypeScript source-map mismatch with Istanbul | Known issue with some TS compilation settings; verify coverage with actual test output |
Specs see no .env values | An older CLI did not load .env for frontmcp test, only for dev | Upgrade the CLI — frontmcp test now loads .env / .env.local with the same precedence as dev (real environment wins, so CI secrets still override). Pass --no-env for a hermetic run |
Invalid first argument, true at collection time | test.skip(condition, reason) reached Jest's skip(name, fn) | Upgrade the CLI — the Playwright signature is supported: test.skip(!hasCredentials, 'credentials not set') skips every test registered after it in the enclosing block |
test.use() in one describe leaks into another, or two files fight over a port | Older @frontmcp/testing kept one global config and per-process ports | Upgrade — test.use() is scoped per describe and ports are locked across Jest workers; use port: 0 |
Every spec fails with HTTP 404 after setting http.entryPath | The test client always connected to the server root | Upgrade the CLI — the client now follows the entryPaths a 404 reports, and test.use({ entryPath: '/mcp' }) sets it explicitly |
test.beforeEach(async ({ mcp }) => …) times out waiting for done | Older @frontmcp/testing handed the hook to Jest, which read the parameter as done | Upgrade — hooks that take a parameter now receive the test's fixtures (same mcp as the test); beforeAll / afterAll never get fixtures |
test.use({ transport: 'sse' }) throws "SSE transport not yet implemented" | Older @frontmcp/testing had no legacy SSE client | Upgrade — 'sse' is the legacy HTTP+SSE transport; the server must enable it with transport: { protocol: { legacy: true } } |
Module @swc/jest in the transform option was not found from frontmcp test | Older @frontmcp/testing did not install the transformer the injected config uses | Upgrade — @frontmcp/testing depends on @swc/jest / @swc/core, and frontmcp test resolves the transformer through it |
ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG from a server started inside a spec | Older SDKs loaded the optional vectoriadb with import() at boot | Upgrade the SDK — vectoriadb is a regular dependency, imported without import(); no --experimental-vm-modules needed |
Each reference has matching examples under examples/<reference>/:
setup-testing| Example | Level | Description |
|---|---|---|
fixture-based-e2e-test | Advanced | Write E2E tests using the fixture API from @frontmcp/testing that manages server lifecycle automatically and uses MCP-specific custom matchers. |
jest-config-with-coverage | Basic | Set up a Jest configuration file that enforces 95%+ coverage across all metrics for a FrontMCP library. |
unit-test-tool-resource-prompt | Intermediate | Write unit tests for the three core MCP primitives, verifying that outputs match the expected MCP response shapes. |
test-auth| Example | Level | Description |
|---|---|---|
oauth-flow-test | Advanced | Use MockOAuthServer to simulate an OAuth identity provider and test the authorization code flow. |
role-based-access-test | Intermediate | Verify that tools enforce role-based access by testing admin and user tokens against protected endpoints. |
token-factory-test | Basic | Use TestTokenFactory to create tokens and verify authenticated and unauthenticated requests. |
test-browser-build| Example | Level | Description |
|---|---|---|
browser-bundle-validation | Basic | Verify that the browser build produces a valid bundle without Node.js-only module references. |
playwright-browser-test | Advanced | Use Playwright to test a browser-based MCP client that loads and calls tools from an MCP server. |
test-cli-binary| Example | Level | Description |
|---|---|---|
binary-startup-test | Basic | Verify that a compiled CLI binary starts correctly and responds to health checks. |
js-bundle-import-test | Intermediate | Verify that the compiled JS bundle can be imported and exports the expected modules after a frontmcp build step. |
test-direct-client| Example | Level | Description |
|---|---|---|
basic-create-test | Basic | Test tools in-memory without any HTTP overhead using the create() function from @frontmcp/sdk. |
openai-claude-format-test | Intermediate | Verify that tools are returned in the correct format for OpenAI and Claude clients using connectOpenAI and connectClaude. |
test-e2e-handler| Example | Level | Description |
|---|---|---|
basic-e2e-test | Basic | Set up a basic E2E test that starts a server, connects a client, and verifies tools are listed. |
manual-client-with-transport | Advanced | Use McpTestClient.create() with explicit transport settings for fine-grained control over E2E tests. |
tool-call-and-error-e2e | Intermediate | Test successful tool calls and verify that invalid inputs produce proper error responses over the full MCP protocol. |
test-tool-unit| Example | Level | Description |
|---|---|---|
basic-tool-test | Basic | Test a simple tool's execute() method with mock context and verify the output. |
schema-validation-test | Intermediate | Validate that a tool's Zod input schema rejects invalid data before execute() is called. |
tool-error-handling-test | Advanced | Test that a tool throws the correct MCP error classes with proper error codes and JSON-RPC error shapes. |
Skills are distributed as plain SKILL.md files plus a sibling references/
and examples/ tree, so consumers can pick whichever access mode fits:
| Mode | How it works |
|---|---|
| Filesystem | Read libs/skills/catalog/frontmcp-testing/ directly from a clone of the catalog repo, or from a published @frontmcp/skills install. SKILL.md is the entry point. |
frontmcp CLI | frontmcp skills list, frontmcp skills read frontmcp-testing, frontmcp skills read frontmcp-testing:references/<file>.md, frontmcp skills install frontmcp-testing — no server required. |
MCP skill:// | When a developer mounts this skill into their own FrontMCP server (@FrontMcp({ skills: [...] })), the SDK exposes it via SEP-2640 resources: skill://frontmcp-testing/SKILL.md, skill://frontmcp-testing/references/{file}.md, etc. The server’s skill://index.json returns the SEP-2640 discovery document for everything mounted on it. |
The catalog itself is not an MCP server. The skill:// URIs only resolve
when a server has been configured to host this skill.
setup-testing, create-tool, create-resource, create-prompt, configure-auth