This is a CLI tool developed for patternfly-react, patternfly-next, and patternfly-org to run axe in Selenium on a list of urls. It has options suited to our needs but should work as an integration test for any project.
It outputs a coverage directory with a webpack-built report, as well as:
- report.json: full axe output per-url
- report.xml: junit coverage grouped into test cases
- report.html: more readable html report of report.xml
patternfly-a11y [command]
patternfly-a11y --help
Commands:
patternfly-a11y <file> Audit a list of URLs in JSON file
Options:
-V, --version output the version number
-c, --config <file> Path to config file
-p, --prefix <prefix> Prefix for listed urls (like https://localhost:9000)
-cr, --crawl Whether to crawl URLs for more URLs (default: false)
--no-code-on-fail By default, exit code 1 is returned on test failures, and 2 if there are incomplete tests, this disables that
-s, --skip <regex> Regex of pages to skip
-a, --aggregate Whether to aggregate tests by component (by splitting URL) in XML report (default: false)
--no-screenshots Whether to save screenshots of visited pages
-ir, --ignore-rules <rules> Axe: Comma-separated list of error ids to ignore (default: "color-contrast")
-iI, --ignore-incomplete Axe: Whether to ignore incomplete errors (default: false)
-t, --tags <tags> Axe: Comma-separated list of accessibility (WCAG) tags to run against (default: "wcag2a,wcag2aa")
-ctx, --context <context> Axe: Context to run in, defaults to document, can be set to a different selector, i.e. document.getElementById("content")
(default: "document")
-h, --help display help for command
yarn add @patternfly/patternfly-a11y
then
yarn exec patternfly-a11y [json-list-of-urls]
OR
yarn exec patternfly-a11y --prefix https://localhost:9000 --crawl /dashboard
For more advanced usage, you can create a config file
yarn exec patternfly-a11y --config a11y-config.js
Sample configuration file (a11y-config.js)
/**
* page: pupeteer page object, for more information see
* https://pptr.dev/#?product=Puppeteer&version=v3.3.0&show=api-class-page
*
* data: contains the options passed in
*/
async function login({ page, data }) {
const user = "user";
const pass = "pass";
await page.waitForSelector("#inputUsername");
await page.type("#inputUsername", user);
await page.type("#inputPassword", pass);
await page.click("button.pf-c-button");
await page.waitForNavigation();
await delay(1000);
}
/**
* page: pupeteer page object, for more information see
* https://pptr.dev/#?product=Puppeteer&version=v3.3.0&show=api-class-page
*/
async function waitForSpinner(page) {
// there should be no loading spinner
await page.waitForSelector(".loading-spinner", {
hidden: true,
});
}
module.exports = {
// the root of the URLs to test
prefix: "https://my-website.com",
// If you need to authenticate/login you can specify a function here
auth: login,
// if there are common elements to wait for on pages to test can specify here
waitFor: waitForSpinner,
// if you want axe to only test a subset of the document you can specify it here, defaults to the entire document
context: 'document.getElementById("page")',
// we define our own list of URLs to test
crawl: false,
// urls can contain strings or objects for more control
urls: [
"/dashboards",
{
url: "/projects",
label: "projects page",
afterNav: async (page) => {
// pupeteer page object, for more information see
// https://pptr.dev/#?product=Puppeteer&version=v3.3.0&show=api-class-page
await page.waitForSelector(".some-element-to-wait-for");
},
},
"/k8s/cluster/projects/default",
"/search/ns/default",
"/api-explorer"
],
};
If updating the source under the report/ directory, create a new webpack build by running yarn build
, which outputs the site to report/dist.
Whenever the patternfly-a11y CLI tool is run, it generates a coverage folder at the root, and copies the report/dist files into it, so that the full report can be served up.
npx serve coverage
To work on and update the react report app, first a coverage/ report needs to exist.
- Run a
./cli.js
CLI command to create the coverage/ report folder. - Run
yarn dev
to launch the react app, it will copy files from the coverage/ report folder into the report/static folder.
PRs and issues are welcome.