Accessibility testing is hard. How do you upskill team members to understand WCAG 2.1? How do you explain testing with screen readers? This is a challenge I faced with new team members and requests to standardise the testing.

I created a checklist which made accessibility testing consistent and removed different interpretations and inaccurate assumptions. When a team's accessibility maturity is low creating a checklist with this methodology results in a testing process anyone can understand.

The testing methodology is comprised of three separate checklists:

1. A WCAG 2.1 level AA testing checklist
2. A compatibility testing checklist
3. A workflow testing checklist

Scope: Individual Pages

For all 50 WCAG 2.1 AA success criterion look at the sufficient techniques and decide if one or more tests can be identified?

For example:

Success Criterion 1.3.1 describes making sure any information and relationship conveyed through presentation is programmatically determinable. Look through the list of sufficient techniques and create a test around the technique.

Applying this technique For Success Criterion 1.3.1 Info and Relationships means at least 13 individual tests can be identified. Your results may be different and may include fewer or greater tests.

Working out your own tests helps you begin to understand what types of elements can be tested in a consistent way.

If a page has the following heading structure:

<h2>Heading 2</h2>
<h4>Heading 4</h4>
<h1>Heading 1</h1>
<h5>Heading 5</h5>

• It will pass the test 'headings are used to structure content'
• It will fail the test 'headings are correctly applied'
• Be recorded as a fail against Success Criterion 1.3.1 Info and Relationships

Whilst headings are applied, they haven't been applied correctly. As the two tests reside under the one success criterion it's recorded as a failure overall.

Looking through the HTML source is slow. Applying Javascript/JQuery bookmarklets make it quicker to identify only the sections you need to test.

e.g. to apply a red 2px solid line to all heading elements: Highlight all headings bookmarklet

if(!($ = window.jQuery)){
  script = document.createElement('script');
  script.src = 'https://ajax.googleapis.com/ajax/libs/jquery/1/jquery.min.js'; 
  script.onload=accessify;
  document.body.appendChild(script);
} 
else{
  accessify();
}
 
function accessify(){
  //place your Javascript/JQuery DOM selection code here
  $('h1,h2,h3,h4,h5,h6').css('outline','2px solid red');
}

To create your own bookmarklets

• Create Bookmarklets - The Right Way
• Bookmarklet Maker

Scope: All representative pages

These are fixed tests covering all representative pages in the site sample which determine how well the content is rendered in browser and screen reader combinations, any irregularities are recorded in each cell. Browser and screen reader combinations need to be tailored for your circumstances.

Each cell records the result of the test YES, NO, N/A. If using Excel the New Comment option on each cell allows a descriptive explanation of the nature of the problem.

Scope: One workflow which may include several pages

This test uses one browser and one screen reader and tests the default path or workflow a user may take through the digital service. Results are recorded via bullet points along with any unusual behaviour, and is based solely on the audible cues from the screen reader.

1. Identify a common workflow or path a user may take through the service
2. Use a browser and screen reader combination which are compatible
3. Avoid looking at the screen
4. Record each step and document behaviour

For example:

Test: Sort all records ascendingly by surname and identify the first record

Steps

1. Navigate to table
2. Navigate to 'surname' column
3. Activate sort control and sort ascendingly
4. Identify the first record in the table

Result

• Can identify 'surname' column
• Can activate sort control
• Applied sort direction is unannounced
• Insufficient guidance how to re-sort column
• Can identify first record

These were conversations I posted to Twitter which resulted in great feedback (click through to read) which helped formulate my approach towards compatibility testing.

I asked on August 27th 2020

It’s become almost subconscious how I accessibility test web sites. But it’s surprisingly difficult to write down or explain, #a11y friends how do you write test cases for screen reader testing for someone new to accessibility?

I asked on August 18th 2020

I've read too many articles that describe how to test with a screen reader but not what to test. The advice of many articles of "have a play around" is not consistent and can't be documented. This is really good from @WeAreSigma describing the what

Testing Accessibility on a Mobile Device – Part 2: Screen reader testing