Skills-SIMarketskills-si.com
Submit a skill
/
The interface is localized. Skill descriptions and documents are machine-translated by AI into your language; code, commands, paths and technical terms stay in the original, and the author's document governs.
Explore skills
Design & creative · SKILL.md

control-ui-e2e

control-ui-e2e

Design, test, fix, or extend OpenClaw Control UI GUI; UI stress-test galleries with feedback inputs, Vitest + Playwright end-to-end checks, mocked Gateway flows, screenshots/videos, agent-verifiable browser proof.

Independent GitHub snapshot · Automated classification · Original documentation governs

Oopenclaw / openclawMIT

Skill documentation

GitHub

name: control-ui-e2e description: Design, test, fix, or extend OpenClaw Control UI GUI; UI stress-test galleries with feedback inputs, Vitest + Playwright end-to-end checks, mocked Gateway flows, screenshots/videos, agent-verifiable browser proof.

Control UI E2E

Control UI design feedback + real browser flows with deterministic Gateway data.

UI Stress Test

  • Substantial UI changes: build local HTML stress-test gallery early; let user compare meaningful states and give feedback on concrete examples.
  • Use for layouts, interactions, components with multiple states. Small copy/icon edits may skip when gallery adds no useful comparison.
  1. Derive examples from affected components + data contracts. Cover relevant normal, loading, empty, error, unavailable, permission, selected, expanded states; long text/dense content where they stress layout. Show mutually exclusive states separately; label proposed states unsupported by current implementation.
  2. One browser-openable HTML overview: stable example IDs, short state labels, enough context to understand each example. Prefer real components + deterministic mock fixtures. Label static/approximate renderings; link running UI for interactions they cannot reproduce. Generated galleries: task-owned artifact storage; no default commit.
  3. Every example: labeled feedback input. Persist locally across refreshes with gallery-specific storage key + stable example IDs. Copy feedback action exports example IDs, state labels, comments as Markdown/plain text for user to return to conversation.
  4. Open gallery in available preview browser; share URL/file path. Preserve same gallery, example IDs, existing comments through iteration. Apply user feedback to both gallery + implementation; keep comparable.
  5. Before requesting feedback: inspect rendered examples at relevant viewport sizes; verify feedback survives refresh + exports correct example labels. Report unsupported states/preview limitations.

Gallery supports design review; never replaces focused behavior tests or inspected before/after proof from running UI. Preserve final proof in fresh capture directories below, separate from evolving gallery.

Test Shape

  • Full GUI flows: ui/src/**/*.e2e.test.ts.
  • ui/src/test-helpers/control-ui-e2e.ts: start Vite Control UI + install mocked Gateway WebSocket.
  • Deterministic scenarios. No live provider keys, real channel credentials, real Gateway unless user explicitly asks for live proof.
  • Narrow rendering logic: prefer existing .browser.test.ts/unit tests. E2E lane: routing, app boot, Gateway handshake, requests, visible UI behavior together.

Commands

  • Target one E2E test in Codex worktree:
node scripts/run-vitest.mjs run --config test/vitest/vitest.ui-e2e.config.ts --configLoader runner ui/src/e2e/chat-flow.messaging.e2e.test.ts
  • Whole local lane in normal checkout:
pnpm test:ui:e2e
  • Use ready dependency installation or prepared normal checkout; never reconcile shared install while other jobs use it.
  • Follow $openclaw-testing: trusted development proof may run locally; remote proof needs browser/platform, clean-environment, or source-isolation reason.

Visual Proof Default

  • Appearance changes: inspected before/after visual evidence.
  • Other behavior: clearest appropriate boundary proof; video + screenshot set not mandatory when assertions already demonstrate change.
  • Deterministic Vitest E2E assertions; never commit generated screenshots/videos.
  • Shared suite disables Chromium partial rasterization + GPU rasterization to avoid observed paint-history-dependent edge pixels. Exact repeat comparisons still required before claiming reproducibility.
  • Transient states such as Saved: install page.clock before fixture; pause with pauseVirtualClock; advance only fixture work needed to enter state. setFixedTime alone does not pause timers. Capture readiness uses native layout delivery without advancing fixture clock.
  • Stills: takeControlUiScreenshotFrame from ui/src/test-helpers/control-ui-e2e-screenshot.ts, explicit semantic content + animations: "disabled". Pass viewport changes/intended scroll target through viewport/scrollTo; verifies retained centering, viewport/scroll-clip intersection, settled layout/fonts/visible images.
  • Pass all related element locators in elements; save returned page PNG + crops from that single frame. Crops enclose fractional bounds in measured PNG pixels; dimensions + unchanged bounds asserted. Never combine separate page/locator screenshots as same-frame evidence.
  • Default current-frame mode + existing viewport/element helpers preserve recording + sampled animation behavior. Static preparation stays active through bounds measurement + unclipped capture; full-page proofs must fit viewport-frame dimensions.
  • After/alongside focused E2E test: run mocked Control UI app when available, e.g. pnpm dev:ui:mock -- --port <port>.
  • Drive Chromium with Playwright against local mock URL. Capture needed states with screenshots/short video.
  • Use browser.newContext({ recordVideo: { dir, size }, viewport }), page.screenshot({ path }); close context before reporting video path.
  • Session-host command-state proof: viewport-only captures, verified with Playwright 1.62.1 + Chrome 151.0.7922.34 (Linux real Gateway; macOS arm64 synthetic reproduction). Other recording owners not migrated/certified by this fix; verify their required screenshot content + finalized video separately. See the verified capture path and upstream limitation.
  • Allocate retained proof with createControlUiE2eArtifactDir(scope, parentDir?) from ui/src/test-helpers/control-ui-e2e-artifacts.ts. Each call atomically creates fresh directory + logs actual path. Parent precedence: explicit parent → trimmed existing OPENCLAW_UI_E2E_ARTIFACT_DIR → repository's .artifacts/control-ui-e2e parent. Existing custom output controls select parents; never add/rewrite env vars to enable capture.
  • Allocate during test/scenario or beforeEach, once per attempt; standalone scripts once per invocation. Pass owner explicitly to shared capture helpers. Keep original gates, feature/stage names, viewports, waits, recording options. Distinct stages: distinct filenames. Keep screenshots/reports/video together.
  • Retain successful + failed evidence. Report actual allocated paths, including relocated filename overrides. Manually delete only exact owned directories after review; never clear shared parents before replay. Disposable build/media fixtures + owned temporary raw video may keep cleanup. New synthetic captures cannot recover overwritten evidence.
  • Timeout diagnostics: fresh children beneath existing diagnostic parent. Mantis retains every capture attempt under invocation-owned directory; refuses report overwrite. Real-Gateway suites, chat-outbox-*, chat-attachment-read-lifecycle: separate owners; coordinate before claiming replay-safe retention there.
  • Recording = validation, not only demo capture. Recorder fails/shows surprising behavior: stop, fix behavior, add/update regression test, rerecord.
  • Visual proof blocked: state exact blocker; still report textual E2E evidence.

Mock Pattern

Start app server; install mock before page.goto; assert Gateway traffic + visible UI:

const server = await startControlUiE2eServer();
const page = await context.newPage();
const gateway = await installMockGateway(page, {
  historyMessages: [{ role: "assistant", content: [{ type: "text", text: "Ready." }] }],
});

await page.goto(`${server.baseUrl}chat`);
await page.locator(".agent-chat__composer-combobox textarea").fill("hello");
await page.getByRole("button", { name: "Send message" }).click();

const request = await gateway.waitForRequest("chat.send");
await gateway.emitChatFinal({ runId: String(request.params.idempotencyKey), text: "Done." });
await page.getByText("Done.").waitFor();

New flow needs more Gateway surface: extend installMockGateway with typed scenario options/method responses.

Run Inspector evidence

  • Collect with withControlUiRunInspector from ui/src/test-helpers/control-ui-run-inspector.ts. Owns separate page; closes on success/failure; leaves caller's Chat page + unsent draft in place.
  • preparePage: mock Gateway or campaign's existing per-tab authentication setup. Shared browser context does not copy another tab's session-storage token.
  • Helper uses rendered component model's RunInspectorSelector + activityRunInspectorSelectorHref, including selected receipt's decision cursor.
  • Rendered panel's data-run-id + data-execution-id: returned present identity, not just requested URL. Selected detail's data-receipt-selector-id = DecisionReceiptDisplayV1.selectorId.
  • Missing/ambiguous identity or missing selected receipts must not count as matches.
  • Decision selectors separate from optional terminal transcript key agent.wait.terminalReceipt.assistantTranscriptIdempotencyKey. Never manufacture key from DOM selector/history row or claim Inspector evidence repairs its absence.
  • Sidebar run state's current accessible label: Active run, not Running.

Standalone Recording

  • Narrated captions, eased target zooms, fast-forwarded pauses: use proof-video dev skill. Standalone template records raw evidence + cue sidecar, renders captioned MP4 with system ffmpeg. Keep raw capture; attach inspected polished video.
  • Already-running mocked Control UI URL: temporary Playwright script or playwright test spec; keep recording flow focused:
    • Open mock URL; interact through stable data-*/user-facing role selectors; wait on asserted states, not fixed sleeps.
    • Request-driven flows: assert visible UI + mocked Gateway traffic. Example: expected count/row visible + sessions.list called with expected search, offset, limit.
    • Short sleeps only after assertions for readable captured video.
    • Generated video: invocation's fresh allocated directory. Never commit or remove older captures.

More skills to explore

View all skills
Saved skills