Checklists are useless. Really?
This week I came across an interesting article on accessibility by Gary Barber. In this article, Gary complains that behind a facade of interest in, and understanding of, web accessibility, organisations in Australia do not really care about making their websites accessible.
I surely agree with him. There are very few organisations that approach us for help with accessibility who are driven by a genuine desire to make their website accessible. In most cases, companies start worrying about accessibility only when they have to meet some legal requirements. The reason behind this is that organisations still do not realise the great benefits (e.g. financial return) of offering accessible websites (sadly, I remember talking about this 6 years ago and little has changed).
I also agree with Gary on the fact that we need a centralised library of accessibility solutions. Whenever I encounter an accessibility issue for the first time, I spend hours reading about different solutions implemented by various developers, trying to decide what would be the most effective one. The creation of a centralised library would make this process much easier.
There is, however, a statement in Gary’s article on which I disagree: “the WCAG checklists don’t work. (…) Checklists are just too open to management and insecure web team members abusing them and warping the real results.” Yes, in an ideal world we would be running accessibility testing with users who have a disability, rather than reviewing websites against checklists. Unfortunately, this is simply not feasible. Due to the variety of types of disabilities that can affect the users’ online experience, and the even larger amount of assistive technologies that people may utilise, a website must be tested with a huge number of users. Even so, it would not be possible to say with any certainty that a website is accessible to everybody. We have tested websites with a sample of people with highly varied disabilities before, and yet at the end of the project we could not be certain that we had covered every issue simply due to the fact that we were catering to such a wide, diverse range of issues. This is why guidelines (like WCAG) and checklists have been written. They are very useful and effective at identifying many common and serious accessibility issues so that they can be resolved. Sure, complying with WCAG does not guarantee that a website is 100% accessible either, but it is a very good starting point to efficiently address many common accessibility issues.
I couldn’t agree more Vlad, the accessibility of websites cannot be assessed using automated testing tools. Understanding the issues that each checkpoint was written to address and the benefits that is meant to bring to users with disabilities is absolutely necessary to decide whether a website satisfies the checkpoint. Clearly, automated testing software cannot do that.
However, in my opinion the origin of the problem does not lie in the fact that checkpoints are not an appropriate mean to evaluate the accessibility of websites. The origin of the problem is, again, that organisations do not understand the importance of accessibility and do not address it seriously.
Many authoring tool vendors have implemented WCAG/Section 508 checklists into accessibility checking features. These checklists give content authors a false sense of accomplishment. The content author’s focus is on doing anything that will pass the checkpoint test rather than real review of content for accessibility issues. Unfortunately, accessibility cannot be packaged into a neat set of automated tests. Let’s take an example. A blank alternate text value will raise an error in accessibility checkers even if it is appropriate for the given image. And if an image contains practically any value for alternate text will cause the checkers to give a passing mark to the checkpoint even if it causes comprehension issues.
I can see where you are coming from with the checklists. And I agree in part. However I have seen these checklists so abused and used as the definitive solution. It would be to the point of 9/10 sites that have had the checklist applied have just glossed over most of the issues that are applicable and ticked them off. We all know the story – If is just becomes to hard to do, the solution is simple, just tick it off – no one will know. After all people using the accessibility components don’t use web sites.
Years ago the accessibility community pointed out that the checklist would be open to abuse. And now I say that we have our evidence.
Yes they are helpful in setting a direction and starting point. But when the completion of the checklist is seen as all that is required you really have to question what is the point.
I don’t have the solution (yet) we need something, maybe a stronger peer review process, that can stop this wholesale checklist abuse.