brainstorming
brainstorming
You MUST use this before any creative work - creating features, building components, adding functionality, modifying behavior, or planning anything new, in software or out of it (a talk, a business, a renovation).
+ `allowJs`) — un-ignoring a file does nothing if no transform matches it |\n| Coverage below 95% | Untested error paths or conditional branches | Run `frontmcp test --coverage` and inspect uncovered lines in the report |\n| E2E test timeout | Server startup too slow or port conflict | Increase Jest timeout; use random port allocation |\n| DI resolution fails in tests | Provider not registered in test scope | Register mock providers before creating the test context |\n| Istanbul shows 0% on async methods | TypeScript source-map mismatch with Istanbul | Known issue with some TS compilation settings; verify coverage with actual test output |\n| Specs see no `.env` values | An older CLI did not load `.env` for `frontmcp test`, only for `dev` | Upgrade the CLI — `frontmcp test` now loads `.env` / `.env.local` with the same precedence as `dev` (real environment wins, so CI secrets still override). Pass `--no-env` for a hermetic run |\n| `Invalid first argument, true` at collection time | `test.skip(condition, reason)` reached Jest's `skip(name, fn)` | Upgrade the CLI — the Playwright signature is supported: `test.skip(!hasCredentials, 'credentials not set')` skips every test registered after it in the enclosing block |\n| `test.use()` in one `describe` leaks into another, or two files fight over a port | Older `@frontmcp/testing` kept one global config and per-process ports | Upgrade — `test.use()` is scoped per `describe` and ports are locked across Jest workers; use `port: 0` |\n| Every spec fails with `HTTP 404` after setting `http.entryPath` | The test client always connected to the server root | Upgrade the CLI — the client now follows the `entryPaths` a 404 reports, and `test.use({ entryPath: '/mcp' })` sets it explicitly |\n| `test.beforeEach(async ({ mcp }) => …)` times out waiting for `done` | Older `@frontmcp/testing` handed the hook to Jest, which read the parameter as `done` | Upgrade — hooks that take a parameter now receive the test's fixtures (same `mcp` as the test); `beforeAll` / `afterAll` never get fixtures |\n| `test.use({ transport: 'sse' })` throws \"SSE transport not yet implemented\" | Older `@frontmcp/testing` had no legacy SSE client | Upgrade — `'sse'` is the legacy HTTP+SSE transport; the server must enable it with `transport: { protocol: { legacy: true } }` |\n| `Module @swc/jest in the transform option was not found` from `frontmcp test` | Older `@frontmcp/testing` did not install the transformer the injected config uses | Upgrade — `@frontmcp/testing` depends on `@swc/jest` / `@swc/core`, and `frontmcp test` resolves the transformer through it |\n| `ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG` from a server started inside a spec | Older SDKs loaded the optional `vectoriadb` with `import()` at boot | Upgrade the SDK — `vectoriadb` is a regular dependency, imported without `import()`; no `--experimental-vm-modules` needed |\n\n## Examples\n\nEach reference has matching examples under [`examples/\u003creference>/`](./examples/):\n\n### `setup-testing`\n\n| Example | Level | Description |\n| ---------------------------------------------------------------------------------------------- | ------------ | ------------------------------------------------------------------------------------------------------------------------------------------------- |\n| [`fixture-based-e2e-test`](./examples/setup-testing/fixture-based-e2e-test.md) | Advanced | Write E2E tests using the fixture API from `@frontmcp/testing` that manages server lifecycle automatically and uses MCP-specific custom matchers. |\n| [`jest-config-with-coverage`](./examples/setup-testing/jest-config-with-coverage.md) | Basic | Set up a Jest configuration file that enforces 95%+ coverage across all metrics for a FrontMCP library. |\n| [`unit-test-tool-resource-prompt`](./examples/setup-testing/unit-test-tool-resource-prompt.md) | Intermediate | Write unit tests for the three core MCP primitives, verifying that outputs match the expected MCP response shapes. |\n\n### `test-auth`\n\n| Example | Level | Description |\n| -------------------------------------------------------------------------- | ------------ | --------------------------------------------------------------------------------------------------------- |\n| [`oauth-flow-test`](./examples/test-auth/oauth-flow-test.md) | Advanced | Use `MockOAuthServer` to simulate an OAuth identity provider and test the authorization code flow. |\n| [`role-based-access-test`](./examples/test-auth/role-based-access-test.md) | Intermediate | Verify that tools enforce role-based access by testing admin and user tokens against protected endpoints. |\n| [`token-factory-test`](./examples/test-auth/token-factory-test.md) | Basic | Use `TestTokenFactory` to create tokens and verify authenticated and unauthenticated requests. |\n\n### `test-browser-build`\n\n| Example | Level | Description |\n| ----------------------------------------------------------------------------------------- | -------- | ------------------------------------------------------------------------------------------------ |\n| [`browser-bundle-validation`](./examples/test-browser-build/browser-bundle-validation.md) | Basic | Verify that the browser build produces a valid bundle without Node.js-only module references. |\n| [`playwright-browser-test`](./examples/test-browser-build/playwright-browser-test.md) | Advanced | Use Playwright to test a browser-based MCP client that loads and calls tools from an MCP server. |\n\n### `test-cli-binary`\n\n| Example | Level | Description |\n| ------------------------------------------------------------------------------ | ------------ | ------------------------------------------------------------------------------------------------------------------ |\n| [`binary-startup-test`](./examples/test-cli-binary/binary-startup-test.md) | Basic | Verify that a compiled CLI binary starts correctly and responds to health checks. |\n| [`js-bundle-import-test`](./examples/test-cli-binary/js-bundle-import-test.md) | Intermediate | Verify that the compiled JS bundle can be imported and exports the expected modules after a `frontmcp build` step. |\n\n### `test-direct-client`\n\n| Example | Level | Description |\n| ----------------------------------------------------------------------------------------- | ------------ | ----------------------------------------------------------------------------------------------------------------------------- |\n| [`basic-create-test`](./examples/test-direct-client/basic-create-test.md) | Basic | Test tools in-memory without any HTTP overhead using the `create()` function from `@frontmcp/sdk`. |\n| [`openai-claude-format-test`](./examples/test-direct-client/openai-claude-format-test.md) | Intermediate | Verify that tools are returned in the correct format for OpenAI and Claude clients using `connectOpenAI` and `connectClaude`. |\n\n### `test-e2e-handler`\n\n| Example | Level | Description |\n| --------------------------------------------------------------------------------------------- | ------------ | -------------------------------------------------------------------------------------------------------------------- |\n| [`basic-e2e-test`](./examples/test-e2e-handler/basic-e2e-test.md) | Basic | Set up a basic E2E test that starts a server, connects a client, and verifies tools are listed. |\n| [`manual-client-with-transport`](./examples/test-e2e-handler/manual-client-with-transport.md) | Advanced | Use `McpTestClient.create()` with explicit transport settings for fine-grained control over E2E tests. |\n| [`tool-call-and-error-e2e`](./examples/test-e2e-handler/tool-call-and-error-e2e.md) | Intermediate | Test successful tool calls and verify that invalid inputs produce proper error responses over the full MCP protocol. |\n\n### `test-tool-unit`\n\n| Example | Level | Description |\n| ----------------------------------------------------------------------------------- | ------------ | -------------------------------------------------------------------------------------------------------- |\n| [`basic-tool-test`](./examples/test-tool-unit/basic-tool-test.md) | Basic | Test a simple tool's `execute()` method with mock context and verify the output. |\n| [`schema-validation-test`](./examples/test-tool-unit/schema-validation-test.md) | Intermediate | Validate that a tool's Zod input schema rejects invalid data before `execute()` is called. |\n| [`tool-error-handling-test`](./examples/test-tool-unit/tool-error-handling-test.md) | Advanced | Test that a tool throws the correct MCP error classes with proper error codes and JSON-RPC error shapes. |\n\n## Accessing This Skill\n\nSkills are distributed as plain SKILL.md files plus a sibling `references/`\nand `examples/` tree, so consumers can pick whichever access mode fits:\n\n| Mode | How it works |\n| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| **Filesystem** | Read `libs/skills/catalog/frontmcp-testing/` directly from a clone of the catalog repo, or from a published `@frontmcp/skills` install. SKILL.md is the entry point. |\n| **`frontmcp` CLI** | `frontmcp skills list`, `frontmcp skills read frontmcp-testing`, `frontmcp skills read frontmcp-testing:references/\u003cfile>.md`, `frontmcp skills install frontmcp-testing` — no server required. |\n| **MCP `skill://`** | When a developer mounts this skill into their own FrontMCP server (`@FrontMcp({ skills: [...] })`), the SDK exposes it via SEP-2640 resources: `skill://frontmcp-testing/SKILL.md`, `skill://frontmcp-testing/references/{file}.md`, etc. The server’s `skill://index.json` returns the SEP-2640 discovery document for everything mounted on it. |\n\nThe catalog itself is **not** an MCP server. The `skill://` URIs only resolve\nwhen a server has been configured to host this skill.\n\n## Reference\n\n- [Testing Documentation](https://frontmcp.dev/reference/testing)\n- Related skills: `setup-testing`, `create-tool`, `create-resource`, `create-prompt`, `configure-auth`\n","licenseDocuments":[]}}
frontmcp-testingUse for anything about testing FrontMCP servers: writing or running unit, integration, and E2E tests and reaching the 95%+ coverage bar. Covers Jest setup and coverage gating; unit-testing a ToolContext execute() with mock context, inputs, and Zod schema validation; testing resources and prompts; in-memory testing via create() and connectOpenAI / connectCla…
Independent GitHub snapshot · Automated classification · Original documentation governs
Entry point for testing FrontMCP applications. This skill helps you navigate testing strategies across component types and find the right patterns for unit, integration, and E2E tests.
setup-testing)frontmcp-development)frontmcp-deployment)Decision: Use this skill for testing strategy and routing. Open the
setup-testingreference underreferences/for hands-on Jest configuration and test writing.
frontmcp-development).setup-testing before opening any other testing skill.This is a router skill. Follow this order to pick a testing approach, then move to the target reference under references/.
references/test-tool-unit.md, references/test-e2e-handler.md, references/test-auth.md) and follow its Steps section.references/setup-testing.md).| Scenario | Reference / Section | Description |
|---|---|---|
| Set up Jest, coverage, and test harness | setup-testing | Full Jest config, test utilities, and coverage thresholds |
| Write unit tests for a tool | test-tool-unit | Mock DI, validate input/output, test error paths |
| Write unit tests for a resource | setup-testing (Unit Testing) | Test URI resolution, template params, read results |
| Write unit tests for a prompt | setup-testing (Unit Testing) | Test argument handling, message generation |
| Write E2E protocol-level tests | setup-testing (E2E Testing) | Real MCP client/server, full protocol flow |
| Test authenticated endpoints | test-auth | E2E with OAuth tokens, session validation, role-based access |
| Test deployment builds | setup-testing + deploy-to-* | Smoke tests against built output |
| Test browser builds | test-browser-build | Smoke-test a frontmcp build --target browser bundle (import the bundle, optional Playwright suite) |
| Test CLI binary builds | test-cli-binary | Spawn-and-curl smoke tests for frontmcp build --target cli artifacts |
| Test with the direct API client | test-direct-client | In-memory testing via create(), connectOpenAI(), connectClaude() (no HTTP) |
| Write E2E test handler patterns | test-e2e-handler | Manual McpTestClient + TestServer E2E patterns (alternative to fixture API) |
| Unit test individual tools | test-tool-unit | Unit testing individual ToolContext subclasses with a mock context |
| Component | Unit Test Focus | E2E Test Focus | Key Assertions |
|---|---|---|---|
| Tool | Input validation, execute logic, error paths, DI mocking | tools/call via MCP client | Output matches schema, errors return MCP codes |
| Resource | URI resolution, read content, template param handling | resources/read via MCP client | Content type correct, URI patterns resolve |
| Prompt | Argument validation, message generation, multi-turn | prompts/get via MCP client | Messages match expected structure |
| Agent | LLM config, tool selection, handoff logic | Agent loop via MCP client | Tools called in order, result synthesized |
| Provider | Lifecycle hooks, factory output, singleton behavior | Indirectly via tool/resource tests | Instance reuse, cleanup on scope disposal |
| Job | Progress tracking, retry logic, attempt counting | Job execution via test harness | Progress events emitted, retries respected |
| Workflow | Step dependencies, conditions, input mapping functions | Workflow run via test harness | Steps execute in order, conditions evaluated, continueOnError respected |
| Skill | Instruction loading (inline/file/url), tool validation | Skill discovery via MCP/HTTP | Instructions resolve, tool refs validated per toolValidation mode |
| Plugin | Context extensions, provider registration, hook firing | Indirectly via tool tests | Extensions available on this, hooks fire at correct stages |
| Pattern | Rule |
|---|---|
| File naming | Always .spec.ts (not .test.ts); E2E uses .e2e.spec.ts |
| File organization | Split E2E tests by app/feature: e2e/calc.e2e.spec.ts, e2e/ecommerce.e2e.spec.ts. Never put all tests in a single server.e2e.spec.ts |
| Environment | frontmcp test loads .env / .env.local into the Jest child, same precedence as dev (real environment wins over the files, config env.shared + env.test underneath). --no-env skips it |
| Test runner | Standalone projects: use frontmcp test (auto-generates Jest/SWC config; discovers src/**/*.spec.ts(x), __tests__/**/*.spec.ts(x), and e2e/**/*.e2e.spec.ts(x); transforms both .ts and .tsx with the automatic JSX runtime; transpiles ESM-only deps such as jose, @noble/hashes and @noble/ciphers (CodeCall) under npm, yarn AND pnpm's node_modules/.pnpm/ store — add your own via test.esmPackages in frontmcp.config.ts; does not force NODE_OPTIONS=--experimental-vm-modules (it would bypass the ESM transforms; scaffolded projects use Jest 30); delegates to a user-provided jest.config.{ts,js,mjs,cjs,json} if present, which drops the injected config unless it starts from preset: '@frontmcp/testing', which carries the same transform, discovery, timeout and ESM transforms). Nx monorepos: use nx test <lib> (resolves the project's jest.config.ts). Never invoke jest --config ... directly |
| Coverage threshold | 95%+ across statements, branches, functions, lines |
| Test descriptions | Plain English, no prefixes like "PT-001"; describe behavior not implementation |
| Mocking | Mock providers via DI token replacement, never mock the framework |
| httpMock scope | httpMock intercepts HTTP in the test process only, NOT in the MCP server subprocess. Do not use httpMock to intercept server-to-API calls — those happen in the child process. Use httpMock for verifying client-to-server request shapes or mocking external APIs called from the test itself |
| Error testing | Assert instanceof specific error class AND MCP error code |
| Async | Always await async operations; use expect(...).rejects.toThrow() for async errors |
| Pattern | Correct | Incorrect | Why |
|---|---|---|---|
| Test file location | fetch-weather.tool.spec.ts next to source | __tests__/fetch-weather.test.ts | Co-location with .spec.ts extension matches FrontMCP conventions |
| DI mocking | Replace token with mock via scope.register(TOKEN, mockImpl) | jest.mock('../provider') module mock | DI mocking is cleaner, type-safe, and tests the real integration path |
| Error assertions | expect(err).toBeInstanceOf(ResourceNotFoundError) | expect(err.message).toContain('not found') | Class checks are stable; message strings are fragile |
| E2E transport | Use @frontmcp/testing MCP client with real server | HTTP requests with fetch | The test client handles protocol details (session, framing) |
| Coverage gaps | Investigate uncovered branches, add targeted tests | Add istanbul ignore comments | Coverage gaps often hide real bugs; ignoring them defeats the purpose |
@frontmcp/testing preset.spec.ts extension throughout| Problem | Cause | Solution |
|---|---|---|
| Jest not finding test files | Wrong file extension (.test.ts instead of .spec.ts) | Rename to .spec.ts; check testMatch in jest.config |
SyntaxError: Unexpected token 'export' | An ESM-only dependency is being ignored instead of transpiled | Add it to test.esmPackages in frontmcp.config.ts. With a hand-written jest.config.ts, start from preset: '@frontmcp/testing' or use the pnpm-safe transformIgnorePatterns in setup-testing AND make sure transform matches .js (^.+\.[tj]sx?$ + allowJs) — un-ignoring a file does nothing if no transform matches it |
| Coverage below 95% | Untested error paths or conditional branches | Run frontmcp test --coverage and inspect uncovered lines in the report |
| E2E test timeout | Server startup too slow or port conflict | Increase Jest timeout; use random port allocation |
| DI resolution fails in tests | Provider not registered in test scope | Register mock providers before creating the test context |
| Istanbul shows 0% on async methods | TypeScript source-map mismatch with Istanbul | Known issue with some TS compilation settings; verify coverage with actual test output |
Specs see no .env values | An older CLI did not load .env for frontmcp test, only for dev | Upgrade the CLI — frontmcp test now loads .env / .env.local with the same precedence as dev (real environment wins, so CI secrets still override). Pass --no-env for a hermetic run |
Invalid first argument, true at collection time | test.skip(condition, reason) reached Jest's skip(name, fn) | Upgrade the CLI — the Playwright signature is supported: test.skip(!hasCredentials, 'credentials not set') skips every test registered after it in the enclosing block |
test.use() in one describe leaks into another, or two files fight over a port | Older @frontmcp/testing kept one global config and per-process ports | Upgrade — test.use() is scoped per describe and ports are locked across Jest workers; use port: 0 |
Every spec fails with HTTP 404 after setting http.entryPath | The test client always connected to the server root | Upgrade the CLI — the client now follows the entryPaths a 404 reports, and test.use({ entryPath: '/mcp' }) sets it explicitly |
test.beforeEach(async ({ mcp }) => …) times out waiting for done | Older @frontmcp/testing handed the hook to Jest, which read the parameter as done | Upgrade — hooks that take a parameter now receive the test's fixtures (same mcp as the test); beforeAll / afterAll never get fixtures |
test.use({ transport: 'sse' }) throws "SSE transport not yet implemented" | Older @frontmcp/testing had no legacy SSE client | Upgrade — 'sse' is the legacy HTTP+SSE transport; the server must enable it with transport: { protocol: { legacy: true } } |
Module @swc/jest in the transform option was not found from frontmcp test | Older @frontmcp/testing did not install the transformer the injected config uses | Upgrade — @frontmcp/testing depends on @swc/jest / @swc/core, and frontmcp test resolves the transformer through it |
ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING_FLAG from a server started inside a spec | Older SDKs loaded the optional vectoriadb with import() at boot | Upgrade the SDK — vectoriadb is a regular dependency, imported without import(); no --experimental-vm-modules needed |
Each reference has matching examples under examples/<reference>/:
setup-testing| Example | Level | Description |
|---|---|---|
fixture-based-e2e-test | Advanced | Write E2E tests using the fixture API from @frontmcp/testing that manages server lifecycle automatically and uses MCP-specific custom matchers. |
jest-config-with-coverage | Basic | Set up a Jest configuration file that enforces 95%+ coverage across all metrics for a FrontMCP library. |
unit-test-tool-resource-prompt | Intermediate | Write unit tests for the three core MCP primitives, verifying that outputs match the expected MCP response shapes. |
test-auth| Example | Level | Description |
|---|---|---|
oauth-flow-test | Advanced | Use MockOAuthServer to simulate an OAuth identity provider and test the authorization code flow. |
role-based-access-test | Intermediate | Verify that tools enforce role-based access by testing admin and user tokens against protected endpoints. |
token-factory-test | Basic | Use TestTokenFactory to create tokens and verify authenticated and unauthenticated requests. |
test-browser-build| Example | Level | Description |
|---|---|---|
browser-bundle-validation | Basic | Verify that the browser build produces a valid bundle without Node.js-only module references. |
playwright-browser-test | Advanced | Use Playwright to test a browser-based MCP client that loads and calls tools from an MCP server. |
test-cli-binary| Example | Level | Description |
|---|---|---|
binary-startup-test | Basic | Verify that a compiled CLI binary starts correctly and responds to health checks. |
js-bundle-import-test | Intermediate | Verify that the compiled JS bundle can be imported and exports the expected modules after a frontmcp build step. |
test-direct-client| Example | Level | Description |
|---|---|---|
basic-create-test | Basic | Test tools in-memory without any HTTP overhead using the create() function from @frontmcp/sdk. |
openai-claude-format-test | Intermediate | Verify that tools are returned in the correct format for OpenAI and Claude clients using connectOpenAI and connectClaude. |
test-e2e-handler| Example | Level | Description |
|---|---|---|
basic-e2e-test | Basic | Set up a basic E2E test that starts a server, connects a client, and verifies tools are listed. |
manual-client-with-transport | Advanced | Use McpTestClient.create() with explicit transport settings for fine-grained control over E2E tests. |
tool-call-and-error-e2e | Intermediate | Test successful tool calls and verify that invalid inputs produce proper error responses over the full MCP protocol. |
test-tool-unit| Example | Level | Description |
|---|---|---|
basic-tool-test | Basic | Test a simple tool's execute() method with mock context and verify the output. |
schema-validation-test | Intermediate | Validate that a tool's Zod input schema rejects invalid data before execute() is called. |
tool-error-handling-test | Advanced | Test that a tool throws the correct MCP error classes with proper error codes and JSON-RPC error shapes. |
Skills are distributed as plain SKILL.md files plus a sibling references/
and examples/ tree, so consumers can pick whichever access mode fits:
| Mode | How it works |
|---|---|
| Filesystem | Read libs/skills/catalog/frontmcp-testing/ directly from a clone of the catalog repo, or from a published @frontmcp/skills install. SKILL.md is the entry point. |
frontmcp CLI | frontmcp skills list, frontmcp skills read frontmcp-testing, frontmcp skills read frontmcp-testing:references/<file>.md, frontmcp skills install frontmcp-testing — no server required. |
MCP skill:// | When a developer mounts this skill into their own FrontMCP server (@FrontMcp({ skills: [...] })), the SDK exposes it via SEP-2640 resources: skill://frontmcp-testing/SKILL.md, skill://frontmcp-testing/references/{file}.md, etc. The server’s skill://index.json returns the SEP-2640 discovery document for everything mounted on it. |
The catalog itself is not an MCP server. The skill:// URIs only resolve
when a server has been configured to host this skill.
setup-testing, create-tool, create-resource, create-prompt, configure-auth