Design Principle
Automated accessibility testing is an early warning system, not complete compliance proof. A healthy process converts tool output into reproducible, fixable, retestable product-task defects.
Large Theory
Accessibility quality cannot be judged by one score or one plugin. Different tools serve different layers: component development, page smoke tests, pre-release audits, human-assisted review, and task-flow verification. Agents should orchestrate them into triage instead of listing tool names in a report.
Small Points
- axe-core is a rules engine that fits Playwright, Jest, Cypress, Storybook, and browser-tool integrations.
- Storybook a11y moves axe checks into component states and design-system previews, but only covers states represented by stories.
- Pa11y fits command-line smoke tests against URLs and local previews, making it useful as a lightweight CI gate.
- Lighthouse accessibility category gives page-level health signals, but a score is not full WCAG coverage.
- Accessibility Insights and WAVE are better for browser-based human-assisted audits that inspect tab stops, structure, accessible names, contrast, and real rendered state.
- Automated tools often cannot fully judge whether copy is meaningful, interaction matches a task, focus order matches business meaning, or screen-reader flow is clear.
Design Judgment
If a tool report only says there are 12 violations, it has not entered the product workflow. It becomes executable triage when each issue is tied to URL/story, state, rule, impact, selector, user task, screenshot, fix recommendation, owner, and retest method.
Implementation Notes
Create an a11y triage record with tool name and version, run time, URL/story, viewport, auth/data state, rule id, impact, selector, screenshot or DOM fragment, user-task impact, fix owner, fix PR, and manual retest result. Let severe rules block CI, move lower-risk issues to backlog, and add manual checks for keyboard path, screen-reader sampling, high contrast, and reduced motion before release.
Counterexamples/Risks
Chasing only a Lighthouse score, running a tool only on the homepage, missing component state stories, ignoring axe incomplete/manual review, treating WAVE as automatic compliance proof, and not retesting the same URL/state after a fix.
Case Study
axe-core positions itself as an accessibility testing engine. Storybook a11y documentation says its addon is powered by axe-core and should act as a first line of QA. Pa11y provides CLI and Node automated testing. Lighthouse covers accessibility, performance, and best practices. Accessibility Insights and WAVE combine automated detection with human evaluation. The reusable lesson is that the toolchain should be layered by development stage, with product defect records as the final output.
Source Links
axe-core: https://github.com/dequelabs/axe-core Storybook accessibility testing: https://storybook.js.org/docs/8/writing-tests/accessibility-testing Pa11y: https://github.com/pa11y/pa11y Lighthouse: https://github.com/GoogleChrome/lighthouse Accessibility Insights for Web: https://github.com/microsoft/accessibility-insights-web WAVE: https://wave.webaim.org/
Agent Directive
After running accessibility tools, convert results into severity-ranked triage records and add manual keyboard, screen-reader, high-contrast, and reduced-motion checks; do not replace accessibility conclusions with a single score.