quality-visual-testingUse this skill when setting up visual testing, visual regression, screenshot diff, Percy, Chromatic, Applitools, or snapshot testing. This skill enforces: tool setup (Percy or Chromatic), baseline management, diff threshold configuration, cross-browser visual testing, and CI integration. Do NOT use for: E2E functional assertions, performance benchmarking, o…
Independent GitHub snapshot · Automated classification · Original documentation governs
Implement visual regression testing with baseline management, diff configuration, and CI integration for pixel-perfect UI verification.
Exact user phrases: "visual testing", "visual regression", "screenshot diff", "Percy", "Chromatic", "Applitools", "snapshot testing", "pixel diff", "visual regression test", "UI diff".
Before activating, verify:
Visual testing setup with tool configuration, baseline management, and CI workflow.
# Tool selection and rationale
# Diff threshold configuration
// CI pipeline configuration
// Baseline management workflow
No preamble. No postamble. No explanations. No filler/hedging/transitions. Compress output -- why use many token when few do trick.
150 lines of configuration and workflow.
Percy: integrates with Playwright, Cypress, Storybook; cross-browser; free tier (5k snapshots/mo). Chromatic: built for Storybook; auto-captures stories; integrates with Git (PR checks); free for public projects. Applitools: AI-powered diffing; ultra-fast grid; advanced layout matching -- best for complex apps but costs more. Recommendation: Percy for E2E visual testing, Chromatic for Storybook-based visual testing.
For team size guidance: 1-5 devs use free tiers (Percy 5K or Chromatic 5K). 5-20 devs use Percy Team ($400/mo) for unlimited snapshots and parallel CI. 20+ devs with Storybook-heavy workflow use Chromatic ($1000/mo). Enterprise with complex apps use Applitools (custom pricing) for AI matching across unlimited resolutions. Open-source or no-budget projects use Playwright built-in screenshot comparison with pixelmatch.
Decision tree:
toHaveScreenshot with pixelmatchPercy + Playwright: npm install @percy/cli @percy/playwright, wrap snapshot calls with await percySnapshot(page, 'Homepage'). Chromatic: npx chromatic --project-token=<token> -- auto-discovers stories.
Playwright built-in: await expect(page).toHaveScreenshot('name.png') with configurable maxDiffPixelRatio and threshold. Store baselines in __screenshots__ directory alongside tests.
// playwright.config.ts
import { defineConfig } from "@playwright/test";
export default defineConfig({
snapshotPathTemplate: "{testDir}/__screenshots__/{testFilePath}/{arg}{ext}",
expect: {
toHaveScreenshot: {
maxDiffPixelRatio: 0.02,
threshold: 0.2,
animations: "disabled",
},
},
});
Per component type: icons and illustrations (0% tolerance), buttons and inputs (0.1%), cards and surfaces (0.2%), images and media (0.5%), full pages (0.3%). Threshold = max diff area as fraction of total pixels. Configure per snapshot: override global threshold when component has intentional dynamic content.
Percy diff levels: "strict" (0% diff), "major" (1% diff), "minor" (5% diff). Use strict for icons and critical UI elements, major for components with some variability, minor for full-page screenshots with dynamic regions. Chromatic uses diffThreshold from 0-1 where 0 is strictest.
Always set per-component thresholds rather than global. Icons and typography should be near-zero tolerance. Media-rich components (images, maps, charts) need higher tolerance for compression artifacts and rendering variation.
Initial baseline capture: run visual tests on main branch, approve all snapshots. New baselines created on every main branch build. PR branches diff against main baseline. Approve/reject diffs via Percy/Chromatic UI. Merging main updates the baseline.
Baseline workflow:
Baseline retention: every main branch build creates a new baseline. Retain last 30 days of baselines. Archive quarterly full-suite snapshots for trend analysis. Purge baselines for deleted components.
For dynamic content: use clip regions to snapshot only static areas, DOM transformation to remove dynamic elements before snapshot, data attribute freeze for loading/error/empty states, and CSS freeze to pause animations before capture.
Configure in Percy dashboard or Playwright projects. Capture snapshots in Chrome (latest), Firefox (latest), Safari (latest). Treat cross-browser diffs as review items -- font rendering varies by OS, accept minor differences.
Cross-browser baseline strategy: Chrome as primary baseline. Firefox shares same baseline with sub-pixel rendering diffs auto-soft-accepted. Safari shares same baseline with font rendering diffs reviewed manually. If using Applitools Ultrafast Grid, configure all target browsers in the Configuration object for parallel cross-browser rendering.
- name: Visual tests
run: npx percy exec -- npx playwright test --grep @visual
env:
PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}
Block PR merge on visual diffs unless reviewed and approved. For parallel builds in Percy, use --parallel-nonce and --parallel-total-to flags. For Chromatic, set --only-changed to skip unchanged stories and --auto-accept-changes for main branch auto-approval. For Playwright built-in, upload diff artifacts on failure for inspection.
--parallel-nonce + --parallel-total-todata-visual-test="skip" attribute on elements to exclude from diffIs the project Storybook-based?
YES -> Does the team need cross-browser?
YES -> Chromatic (built-in cross-browser)
NO -> Chromatic (auto-discovers stories)
NO -> Is the team using Playwright for E2E?
YES -> Percy (native Playwright SDK)
NO -> Is the team using Cypress for E2E?
YES -> Percy (native Cypress SDK)
NO -> Evaluate: BackstopJS (generic), Playwright built-in (free)
Does the project have budget for paid tools?
YES -> Evaluate Applitooks (AI matching, unlimited browsers)
NO -> Playwright toHaveScreenshot or BackstopJS
Does the project need animation diffing?
YES -> Happo (GIF-based diff)
NO -> Standard tools suffice
Is this a main branch build?
YES -> Auto-approve all snapshots -> Update baseline
NO (feature branch) -> Is this the first run?
YES -> No baseline yet, skip diff -> Manual baseline capture needed
NO -> Diff against main baseline
Diff found? -> Is diff expected?
YES -> Approve in dashboard
NO -> Reject, investigate UI change
No diff -> Auto-pass
What component type is being tested?
Icon / Illustration -> threshold: 0 (pixel-perfect required)
Button / Input / Form control -> threshold: 0.001 (minor sub-pixel variation)
Card / Surface / Container -> threshold: 0.002 (shadow rendering variation)
Image / Media / Video player -> threshold: 0.005 (compression artifacts)
Chart / Data visualization -> threshold: 0.003 (rendering engine variation)
Full page / Layout -> threshold: 0.003 (aggregate of all elements)
Third-party embed -> threshold: 0.01 (external content variation) or mask entirely
Does the component contain dynamic content?
YES -> Use clip region, DOM transform, or mask
NO -> Use standard threshold
Using a single diff threshold for all component types causes false positives on strict components (icons) and missed regressions on loose components (images). Configure thresholds per component type or per snapshot.
Capturing screenshots before all resources load, fonts render, or animations complete produces inconsistent baselines. Always wait for network idle, visible content, and animation completion before snapshotting.
Treating all browsers equally without a baseline hierarchy leads to unnecessary review burden. Designate Chrome as primary baseline, auto-soft-accept Firefox sub-pixel diffs, and manually review Safari font rendering diffs.
Date stamps, user avatar images, random IDs, and real-time data cause every snapshot to differ. Use clip regions, DOM transformation callbacks, or data attribute toggling to stabilize dynamic regions before capture.
Approving baseline changes on a feature branch pollutes the baseline with unmerged changes. Always approve diffs post-merge on main to ensure baseline integrity.
Visual testing verifies pixel output but does not validate behavior, accessibility, or performance. Run visual tests alongside functional assertions, accessibility audits, and performance budgets.
Random 1-2 pixel diffs caused by anti-aliasing or sub-pixel rendering create flaky visual tests. Set minimum diff thresholds to absorb rendering noise, and re-run flaky tests before manual investigation.
Running visual tests for every browser and viewport combination in CI can exhaust parallel build slots. Use selective browser testing, parallelization flags, and snapshot filtering to optimize CI resource usage.
Testing only at desktop resolutions misses layout regressions at mobile and tablet breakpoints. Capture snapshots at 375, 768, 1280, and 1920 pixel widths minimum.
When UI changes are intentionally made, developers must update baselines via the tool's UI or CLI. Failing to do so causes every PR to show diffs for the same change, leading to review fatigue.
| Practice | Why | Implementation |
|---|---|---|
| Fixed viewport | Prevents flaky diffs from window size variation | Set viewport in playwright config, never rely on default |
| Disable animations | CSS transitions and JS animations cause false positive diffs | page.addStyleTag({ content: '*, *::before, *::after { animation-duration: 0s !important; transition-duration: 0s !important; }' }) |
| Wait for data | Snapshot before data loads is useless | waitForSelector, waitForResponse, waitForLoadState('networkidle') |
| Consistent fonts | Font loading differences cause OS-level diff variation | Preload web fonts, use font-display: block, or use system font stack |
| Mask dynamic content | Dates, avatars, random data cause every-snapshot diffs | Percy percyCSS with visibility: hidden, Playwright mask option |
| Ignore anti-aliasing | OS-level font rendering produces sub-pixel diffs | Set appropriate threshold values (0.1-0.2%) instead of 0 |
| Test at breakpoints | Responsive design needs multiple screenshots for layout coverage | Test at 375, 768, 1280, 1920 minimum |
| Group by semantic name | Meaningful snapshot names speed up review | Use descriptive names like "Dashboard-metrics-loaded" not "snapshot-1" |
| Run on PR only | Running visual tests on every push wastes CI budget | Run on pull_request event, not push |
| Cache dependencies | Speeds up CI visual test runs | Cache node_modules, Playwright browsers, Storybook build output |
| Preview deployments | Visual tests against live preview environments for accuracy | Deploy preview on Vercel/Netlify, run visual tests against preview URL |
| Review diffs in context | Cloud dashboards with side-by-side view reduce decision time | Use Percy/Chromatic UI review workflows, not raw diff images |
| Approach | Strengths | Weaknesses | Best For |
|---|---|---|---|
| Percy (cloud SaaS) | Native Playwright/Cypress SDK, cross-browser, PR integration, parallel CI | Paid beyond free tier, external service dependency | Teams wanting managed review workflow |
| Chromatic (cloud SaaS) | Storybook-native, auto-discovers stories, Git integration | Storybook-only, no generic E2E visual testing | Storybook-heavy projects |
| Applitools (cloud SaaS) | AI-powered matching, Ultrafast Grid, layout matching modes | Expensive, complex setup | Enterprise cross-browser visual testing |
| Playwright built-in | Free, no external dependency, fast | Manual review, no cloud dashboard, local baseline storage | Teams already using Playwright, no budget |
| Cypress screenshot diff | Free, Cypress-native | Manual review, no cloud dashboard | Teams already using Cypress exclusively |
| BackstopJS (open source) | Free, configurable, Docker support | Manual review, HTML report only, no cloud | Open source projects, no budget |
| Loki (open source) | Docker-based, Storybook integration | Docker dependency, CLI only | Component library visual testing |
| Happo (cloud SaaS) | Animation GIF diff, cross-browser | Paid, smaller ecosystem | Animation-heavy applications |
For most projects: use Percy (Playwright/Cypress) or Chromatic (Storybook). For advanced needs: Applitools. For zero budget: Playwright built-in screenshots. For animation testing: Happo.
Visual testing adds significant time to CI pipelines. Average snapshot capture time: 200-500ms per snapshot depending on page complexity, network requests, and viewport size. A full-page snapshot with 1920px viewport takes longer than an element-level snapshot.
Optimization strategies:
--parallel-nonce + --parallel-total-to, Chromatic parallel builds, Playwright shardingonlyChanged in Chromatic to skip unchanged storiesnetworkIdleTimeout: 100 in Percy config to reduce discovery wait time--project filter to run only desktop snapshots on every PR, mobile snapshots nightlyEach snapshot generates: baseline image + current image + diff image. Average per-snapshot storage: 50-200KB for element snapshots, 200-800KB for full-page snapshots. A suite of 500 snapshots generates approximately 75-400MB per CI run.
Budget considerations:
Visual tests add 2-10 minutes to CI pipeline depending on snapshot count and parallelization level. Without parallelization, 500 snapshots take approximately 8-15 minutes. With parallelization (4 workers), reduce to 3-5 minutes.
Recommended CI optimization:
--grep @visual to tag and filter visual test filespull_request (not on every push)Stringent thresholds (0%) catch all visual changes but increase false positives from anti-aliasing and sub-pixel rendering. Too-loose thresholds (>1%) miss real regressions. Recommended baseline: 0.1% for most components, 0% for critical pixel-perfect elements, 0.5% for media-rich components. Monitor false positive rate weekly and adjust thresholds accordingly.
@percy/cli, @percy/playwright, @percy/cypress, @percy/storybook. PR integration via GitHub/GitLab/Bitbucket apps. Parallel builds via CLI flags. Per-component diff thresholds via Percy config.chromatic CLI, Storybook addon. Git-native review workflow. Auto-accept changes on main branch. TurboSnap for smart snapshot filtering. Zero-config setup for Storybook projects.@applitools/eyes-playwright, @applitools/eyes-cypress, @applitools/eyes-storybook. Ultrafast Grid for parallel cross-browser rendering. AI-powered visual matching with layout, strict, and content match levels.happo-plugin-playwright, happo-plugin-cypress, happo-plugin-storybook. Cross-browser and cross-platform snapshots. Animated GIF diff for motion testing.toHaveScreenshot with maxDiffPixelRatio and threshold options. Baseline storage in repository. No external dependency. Use with @playwright/test for zero-cost visual testing.cy.screenshot() with cy.task for diff comparison. No built-in visual testing -- requires community plugins or custom setup.import { test, expect } from "@playwright/test";
test("product card component renders correctly", async ({ page }) => {
await page.goto("/products/42");
const productCard = page.getByTestId("product-card");
await expect(productCard).toBeVisible();
await expect(productCard).toHaveScreenshot("product-card.png", {
maxDiffPixels: 50,
animations: "disabled",
});
});
test("dashboard page matches baseline", async ({ page }) => {
await page.goto("/dashboard");
await page.waitForLoadState("networkidle");
await expect(page).toHaveScreenshot("dashboard.png", {
fullPage: true,
mask: [
page.getByTestId("user-avatar"),
page.getByTestId("live-timestamp"),
page.getByTestId("third-party-widget"),
],
});
});
import percySnapshot from "@percy/playwright";
test("checkout page visual regression", async ({ page }) => {
await page.goto("/checkout");
await page.waitForLoadState("networkidle");
await percySnapshot(page, "Checkout Page", {
widths: [375, 768, 1280],
minHeight: 2000,
});
});
test("user profile page with stable snapshot", async ({ page }) => {
await page.goto("/profile");
// Freeze dynamic content before snapshot
await page.evaluate(() => {
document.querySelectorAll("[data-dynamic]").forEach((el) => {
el.textContent = "FROZEN_VALUE";
});
});
// Disable animations
await page.addStyleTag({
content: "*, *::before, *::after { animation: none !important; transition: none !important; }",
});
await expect(page).toHaveScreenshot("profile.png");
});
name: Visual Tests
on: pull_request
jobs:
visual:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 20
- run: npm ci
- name: Install Playwright
run: npx playwright install --with-deps chromium
- name: Percy Visual Tests
run: npx percy exec -- npx playwright test --grep @visual
env:
PERCY_TOKEN: ${{ secrets.PERCY_TOKEN }}
PERCY_BRANCH: ${{ github.head_ref }}
PERCY_TARGET_BRANCH: ${{ github.base_ref }}
- uses: actions/upload-artifact@v4
if: failure()
with:
name: visual-diffs
path: __screenshots__/
chromatic:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- run: npm ci
- name: Chromatic Storybook Visual Tests
uses: chromaui/action@v1
with:
projectToken: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}
onlyChanged: true
autoAcceptChanges: false
| Level | Characteristics | Practices |
|---|---|---|
| 1: Initial | No visual testing | Manual visual inspection only, no automation, UI bugs found by users |
| 2: Defined | Basic visual snapshots | Playwright built-in snapshots for critical pages, manual baseline update, local storage |
| 3: Managed | Cloud-based visual testing | Percy/Chromatic with cloud review workflow, per-component thresholds, cross-browser testing, CI integration |
| 4: Measured | Comprehensive visual coverage | Responsive breakpoints tested, dynamic content handling (masks, clips), flaky diff management, quarterly baseline audit |
| 5: Optimized | AI-powered visual QA | Applitools AI matching for layout tolerance, automatic baseline updates on intentional changes, predictive diff analysis, self-healing selectors for visual targets |
Every visual test will fail if it captures date stamps, user avatars, random IDs, or real-time data. You must have a strategy: clip regions, DOM transformation callbacks (percyCSS, page.evaluate), or data attribute toggling to stabilize dynamic regions.
Baseline changes approved on feature branches create baselines contaminated with unmerged changes. When the feature branch is finally merged, the baseline already includes the new look, and the real diff is lost. Always approve baseline changes from main branch builds.
Capturing screenshots before fonts load, images render, or animations complete creates inconsistent baselines. Always wait for networkidle, use waitForSelector for critical elements, and disable animations before capturing.
Different operating systems (macOS vs Linux) and browsers render fonts with different anti-aliasing, creating 1-3 pixel diffs that are not real regressions. Set appropriate thresholds (0.1-0.2%) instead of 0, and use consistent CI environments for baseline and test runs.
baseline_workflow:
capture: "On every main branch build, all snapshots are captured and auto-approved"
review: "On PR builds, diffs are displayed in cloud dashboard for human review"
approve: "Reviewer marks each diff as approved or rejected"
update: "After merge to main, new baseline is automatically created"
retention:
active: "Last 30 days of baselines retained for comparison"
archive: "Quarterly full-suite archive for trend analysis"
cleanup: "Purge baselines for deleted components or retired views"
threshold_strategy:
icons: "0% tolerance — pixel-perfect required"
buttons_inputs: "0.1% tolerance — sub-pixel variation OK"
cards_surfaces: "0.2% tolerance — shadow rendering OK"
images_media: "0.5% tolerance — compression artifacts OK"
full_pages: "0.3% tolerance — aggregate acceptable"
quality-e2e-testing for combined E2E + visual test suite.
design-design-systems for component baseline snapshots.
Carry forward: visual test config, baseline snapshots, review workflow.
` interface EventObserver { onEvent(event: T): Promise; }
class EventBus { private observers: Set<EventObserver> = new Set(); subscribe(observer: EventObserver): void { this.observers.add(observer); } unsubscribe(observer: EventObserver): void { this.observers.delete(observer); } async emit(event: T): Promise { const results = Array.from(this.observers).map(o => o.onEvent(event)); await Promise.allSettled(results); } } `
config: defaults: timeout: 30s retryCount: 3 overrides: production: timeout: 60s retryCount: 5 development: timeout: 300s retryCount: 1
| Metric | Threshold | Severity | Action |
|---|---|---|---|
| Error rate | > 1% over 5min | Critical | Page on-call |
| p99 latency | > 2s over 5min | Warning | Investigate |
| Throughput drop | > 50% over 1min | Critical | Check upstream |
| Queue depth | > 1000 over 1min | Warning | Scale consumers |
| Disk usage | > 85% | Warning | Clean or expand |
| Memory usage | > 90% heap | Critical | Restart or scale |
| Anti-Pattern | Symptom | Root Cause | Solution |
|---|---|---|---|
| Premature optimization | Complex code for no measured benefit | Guessing instead of profiling | Measure first, optimize based on data |
| Copy-paste reuse | Duplicate code across codebase | Lack of abstraction | Extract shared logic into libraries |
| Gold-plating | Features with no current requirement | Over-engineering | YAGNI — build what's needed now |
| Magical thinking | Assumptions without validation | Skipping error handling | Handle all failure modes explicitly |
Cache hierarchy: L1 (in-memory local) → L2 (distributed Redis/Memcached) → L3 (CDN/Edge). Cache invalidation: TTL-based (simple, stale), event-based (complex, fresh), write-through (consistent, higher write latency), write-behind (fast writes, eventual consistency).